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Native Interface Within Games 


Game developers are increasingly applying Java in 
“unpure” ways: using it in conjunction with C/C++ 
code. See how to implement dirty Java techniques, 
and learn how developers like id and Nihilistic 
applied Java to their projects. 

BY BERND KREIMEIER 


40 Optimizing Curved Surface 
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Last month we learned some fundamentals of curved 
surface geometry. This month Sharp explains how to 
optimize Bezier patch processing using a method 
called central differencing, which makes curved sur- 
faces more suitable for real-time 3D graphics games. 
BY BRIAN SHARP 


50 Postmortem: Looking Glass's 
THIEF: THE DARK PROJECT 


One of the most highly acclaimed titles of 1998, THIEF 
almost didn’t make it onto shelves. Despite a fiscal 
crisis at Looking Glass and the loss of important team 
members, the team managed to stay focused and pro- 
duce a game that was a success on numerous fronts. 
BY TOM LEONARD 
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Searching For Answers 


hortly after the horrible 
shooting at Columbine High 
School in Littleton, Colo., I 
picked up The Universe and 
the Teacup: The Mathematics of Truth 
and Beauty, by K.C. Cole. In a chapter 
titled “Calculated Risks,” Cole contrast- 
ed societal fears about airline safety 
whipped up in the aftermath of the 
mysterious crash of TWA Flight 800 
with society’s seemingly resigned 
acceptance that thousands of children 
(equivalent to dozens of filled jumbo 
jets) die every day from malnutrition 
and disease around the world. She mar- 
veled at the way we tune to threats that 
are “exotic, personal, erratic, and dra- 
matic.” And, she noted, that doesn’t 
mean we’re ignorant, “just human.” 

To many, videogames appear exactly 
that: exotic, dramatic, and yes, danger- 
ous. David Grossman, an Arkansas State 
University professor of military science, 
launched a crusade against violent 
videogames, and wants to hold them at 
least partially responsible for tragedies 
like those in Littleton and the 1997 
school shootings in Paducah, Ky. He 
believes that games like QUAKE not only 
influence children, they actually train 
them to shoot. “A hundred things can 
convince someone to want to take a 
gun and go kill,” Grossman said in pre- 
pared testimony to a Senate committee 
this spring. “But only one thing makes 
them able to kill: practice, practice, 
practice.” The implication here is that 
first-person action games provide skills 
necessary to pull off such a shooting. 

This is simplistic nonsense. It’s time 
for pundits like Grossman to quiet 
down until they can submit hard evi- 
dence to support such theories. Accord- 
ing to the New York Times, Mark Manes 
provided Eric Harris and Dylan Klebold 
with the TEC DC-9 used in the crime, 
and the three of them practiced shoot- 
ing in the Colorado mountains prior to 
the incident. There is no evidence that 
any videogame trained the two killers to 
shoot. On the contrary, the facts seem 
to indicate that Harris and Klebold had 
real practice firing their guns. 

As one peels away the layers of these 
cases and looks at the details as they 
emerge, one finds that these incidents 
are not as straightforward as people 
such as Grossman would like the public 
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to believe. As such, I believe that going 
after the producers of videogames, 
movies, and web sites in court will 
prove fruitless. Holding media corpora- 
tions accountable for the acts of men- 
tally ill minors (as 14-year-old Michael 
Carneal pled in the Paducah case) and 
adults on antidepressants (as 18-year- 
old Eric Harris was) is no solution. 
Sadly, it’s just another case of going 
after the deepest pockets. 

Even if you discount any mental ill- 
ness involved, will the deep pockets at 
companies such as Activision (one of 
the defendants named in the Paducah 
case) have to shell out? Some have 
pointed to the recent legal victories over 
tobacco companies as proof that corpo- 
rations can and should be liable for the 
consequences of their products. But the 
gulf between the real dangers of ciga- 
rette smoking and the perceived dangers 
of playing videogames (or watching 
movies or surfing the web) is vast. 
Numerous independent medical studies 
have confirmed the link between smok- 
ing and cancer. Leaked internal docu- 
ments and testimony from former 
tobacco company employees acknowl- 
edged the culpability of tobacco firms 
and helped seal the fate of Big Tobacco. 
No such evidence links videogames and 
violent behavior. This isn’t a simple case 
of cause and effect. 

Looking to the future, as games 
become more realistic and complicated, 
the pleasure of playing videogames may 
become harder to understand by those 
in society less attuned to this form of 
entertainment. They will perceive 
videogames as a growing threat as poly- 
gon counts grow, color depth increases, 
and photo-realistic scenes become nor- 
mal. I don’t think that videogames will 
be as fortunate as comic books, which 
were singled out in the 1950s as a harm- 
ful influence on children, and then 
gradually became accepted as an 
innocuous diversion. Videogames will 
always ride technology’s leading edge, 
and as such, they’re apt to leave many 
paranoid conservatives in their wake. As 
an industry, we have to understand 
that, and be aware that we’ll be under 
the microscope for a long time. & 
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Optimize your code for the latest Intel’ processor with the people who know it best. 


Introducing the award-winning VTune™ Performance Enhancement Environment 4.0, 
the newest comprehensive optimizing solution from Intel. Now programmers can take 
full advantage of the capabilities of today’s Intel” Pentium’ Ill processor. The VTune 
Performance Analyzer, included in the package, provides invaluable software analysis 
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Download a free, 14-day evaluation at: http://developer.intel.com/vtune 
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Play back up to sixteen 24-bit 
digital tracks simultaneously, 
mix with the power of 
aYamaha 02R and add 
professional 32-bit effects 
and EQ without 
ever letting your 
computer Know 
it’s working. 


The Yamaha DSP Factory gives you a constant supply of audio 
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production firepower. The PCI card includes Yamaha-designed chips with 
plenty of muscle to handle all of your recording, mixing, dynamics and 
effects processing needs. In other words, your computer's processor 
never limits the DSP Factory’s extensive capabilities; all it has to 


do is run your favorite recording software. 


The DSP Factory comes with the DS2416 PCI card and stereo analog 
and digital in/out jacks. Expand the system to fit your specific needs 
with the AX44 or the new AX88, which gives you 8 channels of 24-bit 
analog 1/0. However you configure it, Recording magazine says you'll get a 
“serious studio in a box.” And Electronic Musician says “the processing is top- 
notch, the mixer and recorder are extremely powerful...and the whole thing 


sounds great.” Just don’t tell your computer it’s really a music studio. 


@ YAMAHA FACTORY 


For more information, visit www.yamaha.com/proaudio or call (800)937-7171 ext. 632 for literature. 
©1999, Yamaha Corporation of America, Pro Audio Department, P.O. Box 6600, Buena Park, CA 90622-6600 





Does Sex Sell? 


Really, We're Just Friends 


our article “Dolby and Aureal: 


find your magazine to have way too ¥ Contrasts in Audio” (Hard 


many sexual connotations. Are 

these adult games only or do they mar- 
ket to children, too? Advertisements on 
pages 18, 19, 28 and 63 (April 1999) are 
very inappropriate. I understand that 
sex Sells, but isn’t there another way 
companies can think of to make a buck 
and save our society and children from 
negative, wasted energy? Basically, | 
was disgusted by the severity of sexual- 
ity towards women. 

Jennifer Dennis 

via e-mail 


must protest the increasing number 

of advertisements that feature scant- 
ily-clad women and sexual innuendo 
appearing in Game Developer. Looking 
back over the last several issues, the 
number seems to have jumped sharply 
in April. 

The game industry has long had a 


reputation as a boys’ club whichis hos- , 


) 


tile to women. This reputation has 
compromised our ability to attract top 
female talent to our companies, and 
therefore, to make the best games pos- 
sible. Many women developers are 
uncomfortable with the blatant appeals 
to sexuality which are the stock-in- 
trade of our industry’s advertising. This 
material now seems to be creeping 
from the gamers’ magazines into our 
own trade journals, and it sends a dis- 
tinct message that women are not 
wanted or welcome in this business. I 
trust this is not a message that Game 
Developer agrees with. 

These advertisements are insulting to 
our intelligence as developers. No 
developer is so foolish as to make tech- 
nical purchasing decisions on the basis 
of whose ads feature nude women; and 
to assume that we would is to charac- 
terize us as oversexed and stupid. 

As a longtime subscriber to a variety 
of trade journals, I assure you that such 
material is neither ordinary nor appro- 
priate for them. It certainly does not 
appear in Electronic Engineering Times, 
for example, even though its reader- 
ship, like Game Developer’s, is also pre- 
dominantly male. | urge you to reject 
any more advertising on these themes. 

Ernest W. Adams 
Electronic Arts 
Redwood City, Calif. 
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Targets, May 1999) was interesting 
and insightful, and I thank you for 
writing it. 

I am writing to correct you on one 
point you made near the end of the 
article: “Dolby then has to look at DTS, 
and George Lucas’s THX among other 
competitors.” The point I wish to cor- 
rect is regarding THX as 
a competitor to 
Dolby. This is a 
common mis- 
conception that 
THX and Dolby 
are competing 
technologies, when 
in fact they comple- 
ment each other. 

ae THX is a 
@__sihardware certifi- 


) reproduce 
as close as 

cs _} pos sible to the 
original. Dolby SR, Dolby Digital, DTS, 
and SDDS are sound formats that, if 
reproduced on the highest-quality 
THX-certified systems, will produce an 
unsurpassed quality of audio. You are 
correct when you state DTS as compe- 
tition to Dolby. 

Paul Widner 

via e-mail 


Let Him Who Has Played Sin 
Cast the First Stone 


had particular interest in your 

Postmortem on Ritual Entertain- 
ment’s SIN (March 1999). Indeed, the 
column is my favorite in the magazine, 
and I was well aware of the horrible 
bugs in the initial release of SIN. 

In reading the article, I was a little 
shocked that so little mention was 
made of the bugs on release. At the 
very least, they should have been 
under one of the “Things That Went 
Wrong.” There are a few issues sur- 



















Bone to pick? E-mail 
ddunne@sirius.com. Or write to 


Game Developer, 600 Harrison Street, 
San Francisco, CA 94107. 





rounding author Scott Alden’s claims 
that disturb me. 

First, just how bad is their quality 
assurance group? By some reports, the 
game took approximately five minutes 
to load both between levels and when 
restoring a saved game. If the game was 
run from the CD auto-play screen, it 
ran from the CD instead of the hard 
drive, causing save game troubles. 
Saved games themselves were enor- 
mous — people were losing hundreds 
of megabytes from them! There 
was also a bug with the 
first boss encountered, 

which ruined the 
encounter entirely for 
many people. 

These bugs simply 
“slipped through”? 
Second, there was mention of 
the patch, and Activision distributing 
free CDs to all who requested them. I 
do not feel that was an act of charity, 
but an attempt to right a wrong. Some 
of the newsgroup chatter hinted at in 
the article mentioned an apparent 
release agreement with Activision. If 
the game was not out by a certain date, 
Ritual was to suffer a rather large eco- 
nomic penalty. (Presumably, the inten- 
tion was to beat HALF-LIFE to market.) 
Hence, the game shipped early with 
known bugs. 

Hearsay? Conjecture? Or swept 
under the carpet? 





Jose Fernandez 
via e-mail 


RITUAL’S LEVELORD RESPONDS: 
First, you must realize that no game devel- 
oper ever releases bugs intentionally. In the 
case of SIN, we worked very long and hard 
for almost two years. It seemed to be an 
endless crunch time and | nearly had a ner- 
vous breakdown from the long-term stress 
and time demands. | know this seems like a 
dream job, and it is, but | truly almost lost 
my mind from Sin. Losing your mind, by the 
way, /s not fun like some acid trip — it’s 
very scary! 

Anyhow, you must know that we wouldn’t 
just throw something out after all that work. 
If it means anything to say this, we are 
heartbroken that such a cool game as SIN is 
now used asa symbol of what not to do. 
There are other reasons for SIN’s demise. 

Similar to proofreading your own writing 
after you’ve already read it a thousand times, 
or taste-testing a meal after you’ve been in 
the kitchen all day nibbling the ingredients 
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and smelling the seasonings, you really can’t 
effectively and thoroughly verify yourown 
work. You start to see the forest and not the 
trees, and you make too many assumptions 
in the fog of familiarity. You need new eyes 
to truly test anything. We were counting on 
our publisher to do this. 

Being a small development team com- 
pared to most out there, we also lacked 
the person-power to perform complete 
testing. We were counting on our publish- 
er to supplement this need. Each tribe 
member was hurriedly rushing right up 
until the very last moment. Burned out and 
fatigued, it was like trying to herd a dozen 
cats after running a marathon. Remember 
the scene in Jaws in which Richard 
Dreyfuss is frantically tying a buoy to the 
spear gun line and yelling at Robert 
Shawn, “Don’t wait for me!” as he aimed at 
the approaching shark? That’s what it was 
like at the end of SiN. That’s what the end 
of most games Is like. 

We were assured that a large team of 
testers would verify the game and make 
sure it was valid. When we went beta, we 
indeed gota large load of bugs and misbe- 
haviors. This list, however, never seemed 
to change much other than to get smaller 
as we checked off each one. This seems 
obvious now and a warning flag should 
have gone up, but we were too concentrat- 
ed on finishing and burned out from 
exhaustion. It all ended with a “No show- 
stoppers! We’re going gold!” from the pub- 
lisher. We assumed. 

Being small also means that we do not 
have all the varieties of test platforms 
(sound cards, video cards, and so on) to 
test across the board. We also tend to 
develop on our network where slow load 
times are the norm due to pipeline traffic. 
We simply do not have the time to burn CDs 
and load the game on isolated platforms, 
especially towards the end. We were count- 
ing on our publisher to do the hardware- 
related testing, too. 

Please know that we tried very, very hard 
to make SIN the coolest game of 1998. It 
was the coolest, ifnot a good second 
place. Also know that we did everything 
possible to debug the final version and test 
every nuance and permutation of game 
play. Finally, know that no one is hurt more 
by this rush-to-the-shelves calamity than 
we are. After having the privilege of work- 
ing on such excellent games as DUKE NUKEM 
3D and THE SCOURGE OF ARMAGON, It was a 
tremendous blow that SIN was not to follow 
suit merely for reasons of bad management 
and handling. 
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Should the Hall of Fame 
Be the Hall of Games? 


3 t’s revealing sometimes to glimpse 
at what makes people do the 
things they do. Often it’s a complex 
combination of physical, psychologi- 
cal, and yes, spiritual impulses. For 
some reason, we in the entertainment 
industry seem to want to reveal those 
driving influences more than the 
average person does. Maybe it’s 
because we’re in the business of hav- 
ing fun and we think that others 
might want to know why we are in 
this crazy business. Sometimes they 
really do want to know. But I believe 
that mostly they don’t. 

It’s with these thoughts in mind that 
I consider Ernest Adams’s notion of a 
Computer Game Hall of Fame 
(“Immortality for Game Developers,” 
Soapbox, April 1999). While the con- 
cept of a Hall of Fame is intriguing, he 
told me way more than I wanted to 
know. For Ernest, it’s all about immor- 
tality. He wants his own pyramid. Well 
I have news for you, Ernest. The pyra- 
mids may seem immortal, but the 
pharaohs never do. 

Computer gaming is a relatively 
young endeavor compared to other 
pursuits. I like the fact that we don’t 
have a lot of self-appointed experts in 
tweed coats rubbing their chins and 
harrumphing about this or that 
“important” game in the history of 
games. Adams states that the Hall of 
Fame “would be a place where the 
great games are kept, and talked 
about, and studied for the wonder and 
truth that they contain. Above all, it 
would be a place where their designers 
are honored.” Wonder and truth? The 
fact is that the games we make are fun 
and technically amazing and yes, 
worthwhile. But let us not get carried 
away and overstate our contribution 
to mankind. 

Having a place where we can see the 
development of computer games — to 
see the history of computer gaming 
including the designers — is a good 
and interesting idea. A place where we 
mostly raise monuments to the 
builders is less satisfying. | understand 
that Mr. Adams lost a friend and I 
understand wanting to honor that 
friend. I also understand the need for 
recognition and love. But I would pre- 


fer to honor the achievement much 
more than the achievers. After all, as 
Shakespeare might have said, the 
game’s the thing. 
Glenn O’Bannon 
Rainbow Studios 
via e-mail 


n response to Ernest Adams’s 
Soapbox column “Immortality for 
Game Developers” (April 1999), com- 
puter game museums do exist. Look up 
http://www.computerspielemuseum.de 
and http://www.trans-japan.com/ 
vp/bg96 
But the idea is nonsense. Art that 
cannot be experienced is void. We 
don’t need a museum; we need good 
emulators, and we need the greed- 
head companies that crack down on 
them to figure out a way to let them 
thrive. Dani Bunten will be remem- 
bered as more than a marginal figure 
only if future generations can experi- 
ence her games. 
Greg Costikyan 
via e-mail 


ERNEST ADAMS RESPONDS: 
“’homme n’est rien; c’est l’oeuvre qui est 
tout,” as Gustave Flaubert observed to 
George Sand. (“The man is nothing; the 
work is everything.”) Ignoring the author in 
favor of the work is the modus operandi of 
oh-so-trendy postmodern literary criticism, 
and Glenn O’Bannon — his derision for 
intellectuals aside — places himself square- 
ly in that camp. Call me old-fashioned, then, 
but | must disagree. To honor the artist 
along with the art is no more than simple 
justice, as Greg Costikyan eloquently argues 
in his online essay on the subject, which can 
be found at http:/www.crossover.com/ 
costikjustice.html. Should we remember 
Shakespeare’s plays, but forget 
Shakespeare? Remember Mozart’s music, 
but forget Mozart? Remember Spielberg’s 
movies, but forget Spielberg? That would be 
cruelty, indeed. 

If those figures are too grand, then try 
considering my original model, the Pro 
Football Hall of Fame. Can we remember 
Walter Payton’s running without remem- 
bering Walter Payton, or Dick Butkus’s 
tackling without remembering Dick 
Butkus? Should we try? The best computer 
games are expressions of a guiding vision. 
Let us then praise the visionaries along 
with their work. 
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Craig Campanaro, Dec. 1998, www.sharkyextreme.com 


http://www.atitech.com/ad/gd 


© Copyright 1999, ATI Technologies Inc. 
ATI, RAGE 128 and RAGE FURY are 
trademarks and/or regi§tered trademarks 
of ATl Technologies Inc. —s_, 3 





























See nd 10) MURR TO) Be | 


COLOR RESOLUTION PIXEL RESOLUTION FRAME RATE 4 





at 
() sist) pm 












ieave you 
feeling 
satisfied? 































NTRODUCING TRUEMOTION® 2X 
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by Alex Dunne 


SurfaceSuite Pro 1.5 


SVEN TECHNOLOGIES released version 
1.5 of SurfaceSuite Pro, the company’s 
3D texture mapping software which 
lets you apply 2D images to 3D polyg- 
onal, NURBS and patch models. 
Version 1.5 adds two significant new 
features to the product. The first, 
AlphaPaint, lets you paint alpha masks 
onto your textured model, giving you 
more control over texture blending. 
The second addition is PatchWork, 
which lets you create a “quilt” of tex- 
tures — a single skin — for real-time 
polygonal models that you can export 
to your game engine. 

The new version also supports 
Softimage’s .HRC and Rhino’s 3D 
model formats, as well as the .PCX 
image format. Sven perked up the 
product’s interface by adding multiple 
keyboard shortcuts and hotkeys, float- 


Maya 2’s enhancements promise game developers 
improved animation features, including new fur render- 


ing, as seen on this tiger. 


http://www.gdmag.com 





ing windows, automatic layer cre- 
ation, and better support for import- 
ing materials. 

SurfaceSuite Pro supports 3D Studio 
Max, Softimage, Maya, LightWave 3D, 
and Rhino, among other 3D modeling/ 
animation environments. The stand- 
alone product is priced at $595, and 
upgrade discounts are offered. 

@ Sven Technologies 

Palo Alto, Calif. 

(650) 852-9242 

http://www.sven-tech.com 


ALIASIWAVEFRONT introduced Maya 2, 
the latest version of its 3D animation 
and visual effects tool. Perhaps most 
important to game developers, the 
company improved the modeling tools 
— you now have more precise control 
over curve and surface geometry, new 
tools for smoothing polygonal surfaces, 
new texturing tools and the ability to 
assign arbitrary data to polygon ver- 
tices. With the Maya Unlimited pack- 
age, you get more advanced modeling 
tools (including sup- 
port for subdivision 
Surfaces) as well as fur 
and cloth animation 
features. 

In the area of char- 
acter modeling and 
animation, Maya 2 
has new deformer 
types and automated 
skinning capabilities 
for faster creation and 
easier control of com- 
plex characters. The 
tool uses a pose-based 
approach that lets 
you treat a complex 
character as a single 
entity as you animate 
it. Alias|Wavefront 
also souped up 
Maya’s renderer, 


New Products: Sven Technologies 

updates SurfaceSuite Pro, Alias|Wave- 
front rolls out Maya 2, and OpenGL 

arrives for Macintosh. p. 9 


Industry Watch: Sega reveals pricing 
for Dreamcast, EA hits the billion-dollar 
mark, Activision squeaks by, and 
Interplay faces red ink. p. 10 


which can speed rendering up by 90 
percent in some cases, according to the 
company. Maya 2 also supports multi- 
threaded batch rendering, too. 

Maya Complete 2 is priced at $7,500 
and includes modeling, rendering, ani- 
mation, dynamics, Artisan and MEL 
(the tool’s embedded scripting lan- 
guage) features. Maya Unlimited 2 
costs $16,000 and includes the features 
found in the Complete version, plus 
Maya Live, Maya Fur, Maya Cloth and 
new advanced modeling features. Both 
Maya flavors are available for Windows 
NT and IRIX. 

@ Alias|Wavefront 

Toronto, Ont., Canada 

(416) 362-9181 

http: //www.aw.sgi.com/entertainment 


OpenGL for the Macintosh 


APPLE recently shipped OpenGL for 
Macintosh, further illustrating the way 
in which Steve Jobs is shifting Apple’s 





focus back towards games. Available 
freely from the Apple web site for 
download, OpenGL for Macintosh 
brings the industry-standard 3D API to 
the Mac. OpenGL for Macintosh 
requires a PowerPC-based Macintosh 
computer with MacOS 8.1 or later. An 
iMac or new Power Macintosh G3 is rec- 
ommended for accelerated 3D render- 
ing. The version available from Apple’s 
web site includes libraries to accelerate 
rendering on Rage II, Rage Pro, and 
Rage 128-based Macintosh systems. 
@ Apple Computer 

Cupertino, Calif. 

(408) 996-1010 

http://www.apple.com/opengl 
1999 


A GAME DEVELOPER 

















@ Industry Watch 


by Alex Dumne 


DISNEY INTERACTIVE AND NINTENDO 
announced a new partnership, which 
will introduce Mickey Mouse to the 
world of 3D games. Nintendo will 
publish a number of Mickey Mouse- 
based games for the N64 and the 
Game Boy Color, to be developed by 
Rare Ltd. Two of the titles will be rac- 
ing games for the N64 and Game Boy 
Color (coming out in 1999 and 2000), 
and the third will be a “Mickey 
Adventure” title, slated for release in 
2001 on Nintendo’s next console sys- 
| tem and the Game Boy Color. 
Additionally, as part of the agreement, 
Disney Interactive will develop multi- 
ple titles for the Game Boy Color 
aimed at the girl-games market. These 
titles will be based upon Disney’s 
Beauty and the Beast and Alice in 
Wonderland movies. 


SPEAKING OF THE GAME BOY COLOR, 
Nintendo said that sales of the Game 
Boy Color during the first quarter of 


Macworld Expo 


JACOB J. JAVITS 
CONVENTION CENTER 
New York, NY 
July 21-23, 1999 
Cost: $45-$1,195 = 


http://www.macworldexpo.com 
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August 8-13, 1999 
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1999 averaged 94,000 units per 
week, compared to 31,000 in 
1998 for the previous Game Boy 
unit. Peter Main, Nintendo’s 
executive vice president of sales 
and marketing, said that since 
the launch of the Game Boy 
Color in the U.S. last November, 
the company has sold more 
than two million units. 


ATOMIC POWER AT MINDSCAPE. 
Mindscape signed a publishing 
deal with Atomic Games, which 
lets the ‘Scape take over the 
successful CLOSE COMBAT series 
from former publisher 

Microsoft. The deal adds another 
strong war game franchise to Mind- 
scape’s lineup, which already included 
PANZER GENERAL and STEEL PANTHERS. The 
next game scheduled for the CLOSE 
ComBAT Series is slated for release 
sometime in the fourth quarter. 


ACTIVISION BARELY BEATS ESTIMATES. 
Activision’s fourth fiscal-quarter (ending 
March 31) profits were a bit better than 
analysts had expected, amounting to 
$5.2 million, which was an improve- 
ment over the $689,000 that the com- 
pany earned last year during the same 
period. CEO Bobby Kotick credited the 
company’s wide array of games for 
boosting the company’s market posi- 
tion, and indicated that the company 
was on the prowl for large acquisition 
targets. However, CFO Barry Plaga pre- 
dicted Activision would post a loss in 
the first fiscal quarter of 1999, as it did 
last year. 


ARCADE DOWSER. In an attempt to 
counteract the shrinking arcade market 
and make it easier to bag that increas- 
ingly elusive quarry, the loyal arcade 
gamer, Midway Games and WMS 
Industries launched a web-based data- 
base application that tells you exactly 
where you can find Midway coin-op 
games around the world. This simple 
game finder, available on on Midway’s 
website (http://www.midway.com), also 
lets visitors submit new locations when 
they find games. Planning a trip to 
Madagascar? Need to satisfy your 
CARNEVIL fix while you’re there? Now 
you know where go... 


MUSIC TO THEIR EARS? Aureal Semi- 
conductor, maker of the Vortex2 digital 


Mindscape is adding the Close Comsat series to 
its arsenal of war games. 





audio processor and the A3D audio 
API, announced financial results for 
the first fiscal quarter ending April 4, 
1999. Revenues reached $12.6 million, 
amounting to a 250 percent increase, 
but saw a net loss of $4.1 million (bet- 
ter than its $5.5 million loss last year). 
What looks most promising is that the 
company’s gross margins grew to 34 
percent in the first quarter from 22 
percent last year, thanks to increased 
demand for sound cards based on the 
Vortex2. 


DREAMCAST PRICING AT LAST. Under- 
cutting industry estimates, Sega’s Bernie 
Stolar revealed that the the Dreamcast 
suggested launch price would be $199. 
Additionally, Stolar confirmed that the 
U.S. launch date would be September 9, 
1999. There will be between 10 and 12 
titles available around launch time, 
plus more than 20 first-party titles on 
track for the year 2000. To date, retail- 
ers have placed pre-orders for 30,000 
systems, according to Sega. 


$109. EA released its fiscal year results. 
Two words: banner year. Congratula- 
tions go out to Larry Probst and com- 
pany, for being the first entertainment 
software publisher to top a billion dol- 
lars in revenue. 


NEWS FROM THE LAND OF RED INK. 
Poor Interplay continues to have a 
tough time. The company lost $8.28 
million in its first quarter, compared to 
a profit of $3.1 million a year ago. 
Interplay blamed its larger-than-expect- 
ed loss on the fact that it hasn’t 
released any major titles recently, and 
the impact of higher-than-anticipated 
product returns. @ 
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by Jeff Lander | 





Flex Your Facial 


Animation Muscles 





| It may seem as if this is purely an art 
_| problem, better left to your art staff (see 
“ | Artist’s View this month, “Talking 


Heads: Hierarchical Animation in Real- 
time 3D”). Or, if you are a one-person 
development team, at least left to the 
creative side of your brain. However, 
your analytical side needs to inject itself 
in here a bit. This is one of those early 
production decisions you read about so 
much in the Postmortem column that 
can make or break your schedule and 
budget. Choose wisely and everything 
will work out great. Choose poorly and 
your art staff or even your own brain 
will throttle you. 


Decisions, Decisions 


= or the final result, | want a 3D real- 
time character that can deliver vari- 
ous pieces of dialog in the most con- 
vincing manner possible. Thanks to the 
information learned last month, I know 
I can severely limit the amount of work 
I need to do. I know that with 13 
visemes, or visual phoneme positions, I 
can reasonably represent most sounds | 
expect to encounter. I even have a nice 
mapping from American English to my 
set of visemes. Most other languages 
could probably be represented by these 
visemes as well, but could require a dif- 
ferent mapping table. 

From this information I can expect 
that if I can reasonably represent these 
13 visemes with my character mesh, 


visemes in a real-time application. 


then continuous lip-synch should be 
possible. So the problem really comes 
down to how I construct and manipu- 
late those meshes. 


Viseme-Based Methods 


Cc ertainly, the obvious method for 
creating these 13 visemes is to gen- 
erate 13 versions of my character head 
mesh, one to represent each viseme. | 
can then use the morphing techniques I 
discussed in last December’s column 
(“Mighty Morphing Mesh Machine,” 
December 1998) to interpolate smooth- 
ly between different sounds. 

Modeling the face to match the 
visemes is pretty easy. Once the artist 
has the base mesh created, each viseme 
can be generated by deforming the 
mesh any way necessary to get the 
right target frame. As long as no ver- 
tices are added or deleted and the trian- 
gle topology remains the same, every- 
thing should work out great. Figure 1 
shows an image of a character display- 
ing the “L” viseme, as in the word 
“life.” The tongue is behind the top 
teeth, slightly cupped, leaving gaps at 
the side of the mouth, and the teeth 
are slightly parted. 

Sounds pretty good so far. Just create 
13 morph targets for the visemes in 
addition to the base frame and you’re 
done. Life’s great, back to physics, 
right? Well, not quite yet. 

Suppose in addition to simply lip- 
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ast month I left off with a nice short list of the visemes I would need to 
represent speech realistically. However, now I am left with the not 


insignificant problem of determining exactly how to display these 


synching dialog, your characters must 
express some emotion. You want them 
to be able to say things sadly, or speak 
cheerfully. We need to add an emo- 
tional component to the system. 


Adding Some Heart to the Story 


A! first glance, it may seem that 
you can simply add some addi- 
tional morph targets for the base emo- 
tions. Most people describe six basic 
emotions. Here they are with some of 
their traits. (See Goldfinger under “For 
Further Info” for photo examples of 
the six emotions.) 

1. HAPPINESS: Mouth smiles open or 
closed, cheeks puff, eyes narrow. 

2. SADNESS: Mouth corners pull down, 
brows incline, upper eyelids droop. 

3. SURPRISE: Brows raise up and arch, 
upper eyelids raise, jaw drops. 

4. FEAR: Brows raise and draw togeth- 
er, upper eyelids raise, lower eyelids 
tense upwards, jaw drops, mouth cor- 
ners go out and down. 

5. ANGER: Inner brows pull together 
and down, upper eyelids raise, nostrils 
may flare, lips are closed tightly or 
open exposing teeth. 

6. Discust: Middle portion of upper 
lip pulls up exposing teeth, inner brows 
pull together and down, nose wrinkles. 

There are variations of these emo- 
tions, such as contempt, pain, distress, 
excitement, but you get the idea. Very 
distinct versions of these six will get 
the message across. 

The key thing to notice about this 
list is that many of these emotions 
directly affect the same regions of the 
model as the visemes. If you simply 
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FIGURE 1. The “l” viseme as seen at 
the start of the word “life.” 





layer these emotions on top of the 
existing viseme morph targets, you can 
get an additive effect. This can lead to 
ugly results. 

For example, let me start with the 
“L” sound from before and blend in a 
surprised emotion at 100 percent. The 
“L” sound moves the tongue up to the 
top set of teeth and parts the mouth 
slightly. However, the surprise target 
drops the jaw even farther but leaves 
the tongue alone. This combination 
blends into the odd-looking character 
you see in Higure 2, 

This problem really becomes appar- 
ent when the two meshes are actually 
fighting each other. For example, the 
“oo” viseme drives the lips into a tight, 
pursed shape while the surprise emo- 
tion drives the lips apart. Nothing pret- 
ty or realistic will come out of that 
combination. 

When I ran into this issue a couple 
of years ago, the solution was tied to 
the weighting. By assigning a weight or 
priority to each morph target, I can 
compensate for these problems. I give 
the “oo” viseme priority over the sur- 
prise frame. This will suppress the 
effect that the surprise emotion has 
over shared vertices. 


Welcome to Muscle Beach 


ost of the academic research on 
facial animation has not 
approached the problem from a viseme 
basis. This is due to a fundamental 
drawback to the viseme frame based 
approach. In the viseme-based system, 
every source frame of animation is 
completely specified. While I can speci- 
fy the amount each frame contributes 
to the final model, I cannot create new 
source models dynamically. Say, for 
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FIGURE 2. Avery surprised “I” 
viseme. 





example, I want to allow the character 
to raise one eyebrow. With the frames I 
have described so far, this would not be 
possible. In order to accomplish this 
goal, | would need to create individual 
morph targets with each eyebrow 
raised individually. Since a viseme can 
incorporate a combination of many 
facial actions, isolating these actions 
can lead to an explosive need for 
source meshes. You may find yourself 
breaking these targets into isolated 
regions of the face. 

For this reason, researchers such as 
Frederic Parke and Keith Waters began 
examining how the face actually works 
biologically. 
By examining 
the muscle 
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ble.. In Tact, 
psychologists 
Paul Ekman 
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these action units that can create thou- 
sands of facial expressions. By creating a 
facial model that is controlled via these 
action units, Waters was able to simu- 
late the effect that changes in the action 
units reveal on the skin. 

While I’m not sure if artists are ready 
to start creating parametric models 
controlled by virtual muscles, there are 
definitely some lessons to be learned 
here. With this system, it’s possible to 
describe any facial expression using 
these 50 parameters. It also completely 
avoids the additive morph problem | 
ran into with the viseme system. Once 
a muscle is completely contracted, it 
cannot contract any further. This limits 
the expression to ones that are at least 
physically possible. 


Artist-Driven Muscle-Based Facial 
Animation 


nimation tools are not really 
developed to a point where artists 
can place virtual muscles and attach 
them to a model. This would require a 
serious custom application that the 
artists may be reluctant even to use. 


However, that doesn’t mean that these 
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Mentalis 


the measure- 
ment of indi- 
vidual muscle 
groups as 
“action units.” 
Their system, 
called Facial 
Action Coding 
System 
(FACS), 
describes 50 of 
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CHART 1. The basic muscle groups involved in facial animation. 
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Zygomaticus 
Major 


FIGURE 3. The zygomaticus major 
muscCcle will put a smile on your face. 





methods are not available for game 
production. It just requires a different 
way of thinking about modeling. 

For instance, let me take a look at 
creating a simple smile. Biologically, I 
smile by contracting the zygomaticus 
major muscle on each side of my face. 
This muscle connects the outside of the 
zygomatic bone to the corner of the 
mouth as shown in Figure 3. Contract 
one muscle and half a smile is born. 

O.K. Mr. Science, what does that 
have to do with modeling? Well, this 
muscle contracts in a linear fashion. 
Take a neutral mouth and deform it as 
you would when the left zygomaticus 
major is contracted. This mesh can be 
used to create a delta table for all ver- 
tices that change. Repeat this process 
for all the muscles you wish to simulate 
and you have all the data you need to 
start making faces. You will find that 
you probably don’t need all 50 muscle 
groups described in the FACS system. 
Particularly if your model has a low 
polygon count, this will be overkill. 
The point is to create the muscle 
frames necessary to create all the 
visemes and emotions you will need, 
plus any additional flexibility you 
want. You will probably want to add 
some eye blinks, perhaps some eye 
shifts, and tongue movement to make 
the simulation more realistic. 

The FACS system is a scientifically- 
based general modeling system. It does 
not consider the individual features of 
a particular model. By allowing the 
modeler to deform the mesh for the 
muscles instead of using this algorith- 
mic system, I am giving up general 
flexibility over a variety of meshes. 
However, I gain creative control by 
GAME DEVELOPER 
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FIGURE 4. Pucker up: Incisivus labii 
at work. 





allowing for exaggeration as well as 
artistic judgement. 

The downside is that it is now much 
harder to describe to the artists what it 
is you need. You need to purchase 
some sort of anatomy book (see my 
suggestions at the end of the column) 
and figure out exactly what you want 
to achieve. Your artists are going to 
resist. You had this nice list of 13 
visemes and now you are creating more 
work. They don’t know what an inci- 
sivius labii is and don’t want to. You 
can explain that it is what makes Lara 
pucker up and they won't care. You 
will have to win the staff over by show- 
ing the creative possibilities for charac- 
ter expression that are now available. 
They probably still won’t care, so get 
the producer to force them to do it. I 
have created a sample muscle set in 
Chart 1. This will give you some groups 
from which to pick. 

Now I need to relate these individual 
muscle meshes to the viseme and emo- 
tional states. This is accomplished with 
“muscle macros” that blend the per- 
centages of the 
basic muscles to 
form complex 
expressions. 
This flexibility 
permits speech 
and emotion in 
any language 
without the 
need for special 
meshes. 

[ still need to 
handle the case 
where several 
muscles inter- 
act with the 


FIGURE 5. 


“oo” viseme. Image courtesy of Virtual Celebrities Productions 
and Quantumworks. 


Same vertices. However, now there is a 
biological foundation to what you are 
doing. 

Certain muscles counteract the 
actions of other muscles. For example, 
the muscles needed to create the “oo” 
viseme (incisivius labii) will counter the 
effect of the jaw dropping (digastric for 
those of you playing along at home). 
One real-time animation package I 
have been working with called 
Geppetto, from Quantumworks, calls 
this Muscle Relations Channels. You 
can create a simple mathematical 
expression between the two to enforce 
this relationship. You can see this 
effect in Figure 3. 


finally have my system set up and 

my models created. It is time to cre- 
ate some real-time animation. The 
time-tested animation production 
method is to take a track of audio dia- 
log and go through it, matching the 
visemes in your model set to the dia- 
log. Then, in second pass, go through it 
and add any emotional elements you 
want. This, as you can imagine, is pret- 
ty time consuming. Complicating the 
matter is that there are not many off- 
the-shelf solutions to help you out. The 
job requires handling data in a very 
special way and most commercial ani- 
mation packages are not up to the task 
without help. 

Detecting the individual phonemes 
within an audio track is part of the 
puzzle that you can get help with. 
There is an excellent animation utility 
called Magpie Pro from Third Wish 
Software that simplifies this task. It can 
take an audio track and analyze it for 
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phoneme patterns you provide auto- 
matically. While not entirely accurate, 
it will at least get you started. From 
there you can manually match up the 
visemes to the waveform until it looks 
right. The software also allows you to 
create additional channels for things 
such as emotions and eye movements. 
All this information can be exported as 
a text file containing the transition 
information. This in turn can be con- 
verted directly to a game-ready stream 
of data. You can see Magpie Pro in 
action in Figure 6. 


Wire Me Up, Baby 


—~_ WW ith all the high-tech toys avail- 
) able these days, it may seem 


like a waste to spend all this time 
hand-synching dialog. What about 
this performance capture everyone has 
been talking about? There are many 
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facial capture 
devices on the 
market. Some 
determine facial 
movements by 
looking at dots 
placed on the 
subject’s face. 
Others use a 
video analysis 
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mation on this 
aspect, have a 
look at Jake 
Rodgers’s article 
“Animating 
Facial Expres- 
sions” (November 1998). The end 
result is a series of vectors that describe 
how certain points on the face move 
during a capture session. The number 
of points that can be captured varies 
based on the system used. However, 
typically you get from about eight to 
hundreds of sensor positions in either 
2D or 3D. The data is commonly 
brought into an animation system like 
Softimage or Maya and the data points 
drive the deformation of a model. 
Filmbox by Kaydara is designed specifi- 
cally to aid in the process of capturing, 
cleaning up, and applying this form of 
data. Filmbox can also apply suppres- 
sive expressions, inverse kinematic 
constraints, and perform audio analy- 
sis similar to Magpie Pro. 

This form of motion capture clearly 
can speed up the process of generating 
animation information. However, it’s 
geared much more toward traditional 
animation and high-end performance 
animation. In this respect it doesn't 
really suit the real-time game develop- 
er’s needs. It’s possible to drive a real- 
time character by using the raw 
motion capture data to drive a facial 
deformation model. However, fora 
real-time game application, I do not 
believe this is currently feasible. 

In order to convert this stream of 
positional data into my limited real- 
time animation system, | would need to 
analyze the data and determine what 
visemes and emotions the performer is 
trying to convey. You need a filtering 
method that will take the multiple sam- 
ple points and select the viseme or MUusS- 


FIGURE 6. Magpie Pro simplifies the task of isolating 


phoneme patterns in your audio track. 


cle action that is occurring. This is really 
the key to making motion capture data 
usable for real-time character anima- 
tion. This area of research, termed ges- 
ture recognition, is pretty active right 
now. There is a lot of information out 
there for study. However, Quantum- 
works’s Geppetto provides gesture 
recognition from motion capture data 
to drive “muscle macros” as both a 
standalone and a plug-in for Filmbox. 


Where Do We Go from Here? 


etween viseme-based and muscle- 

based facial animation, you can 
see that there are a lot of possible 
approaches and creative areas to 
explore. In fact, the whole field has 
really opened up to game development 
in terms of opportunities for game pro- 
ductions as well as tool developers. 
Games are going to need content to 
start filling up those new DVD drives 
and I think facial animation is a great 
way to take our productions to the 
next level. 
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Talking Heads: Hierarchical Facial 
Animation in Real-Time 3D 





time 3D entertainment. 

This month’s article is the first 
installment in a series dedicated to 
expanding the knowledge base of real- 
time facial animation. Over the next 
few months, we'll discuss facial anato- 
my and skinning techniques, as well 
as phoneme recognition and anima- 
tion with linked expressions. We'll 
tackle the problem from the ground 
up, and by the end of the process we'll 
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FIGURE 1. Real-time head and face. 


ntil very recently, facial a 
motion video and prerendered 


rendering technology and geometry-dedi 


the door for animators to use this technique within the realm of real- 


have covered all of the steps necessaLy 
to create and animate a speech-driven 
human head. 


Why Go to the Trouble? 


here is absolutely no logical rea- 

son why the actors in today’s real- 
time 3D games shouldn't be able to 
smile, scowl, and talk with the player. 





nimation was a technique reserved for full 
cinematics. The continuing advances in 


cated processors have opened 


The struggle to create a believable vir- 
tual world is the struggle to create the 
illusion of reality. For any game based 
within a virtual environment, the 
player’s enjoyment Is directly linked 
to how immersive that environment 
feels. In the ideal case, players will for- 
get that they are sitting in front of a 
computer screen, and will lose them- 
selves for a few hours within the virtu- 
al worlds we create. It is incumbent 
upon us as developers to use every 
means at our disposal to generate this 
effect. Facial animation can and will 
be one of the most effective tools for 
achieving this. When in-game charac- 
ters interact with the player through 
recognizable facial expressions and 
lip-synched spoken dialogue, we will 
have taken several steps towards 
achieving the perfect virtual world. 


Modeling the Head and Face 


he main reason we haven't been 
able to create believable lip- 
synched characters has been the ren- 
dering engines’ polygon limitation. In 
4 human head, the skin of the face is 
very plastic, in other words, extremely 
malleable. In order to achieve this 
effect convincingly and without 
excessive distortion, the polygonal 
densities of the face are required to be 
fairly high. As a result of the recent 
improvements in rendering and pro- 
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cessing power, and through aggressive 
use of level-of-detai] models, most cut- 
ting-edge engines will now accommo- 

date the relatively high density mesh- 

€s required for facial animation. 


result, setting up a skeletal hierarchy 
at the beginning can still get you 
there because the mesh that’s pro- 
duced for each skeletal animation can 
then be cloned and used as a morph 


The main reason we haven’t been able to 
create believable lip-synched characters 
has been the rendering engines’ polygon 


limitation. 


For our example we'll use a human 
head, although the basic principles dis- 
cussed will apply to almost any face 
with the basic bilateral construction 
found in humans. The head in Figure ] 
has been modeled with sufficient poly- 
gons for facial expression, but still low 
enough for current engine technology. 
Almost all of the head’s 800 or SO poly- 
gons have been devoted to the areas 
surrounding the eyes, nose, and 
mouth, where the bulk of facial expres- 
sion is displayed. In this Case, we’ve 
actually gone to the trouble to add the 
internals of the mouth, with tongue, 
teeth, and cheek surfaces included (this 
will be necessary if our character is 
going to be speaking close up, in an in- 
same Cut-scene, for example). 

Although this head has been mod- 
eled with real-life photographic refer- 
ence, which is often the best resource, 
Hogarth’s Drawing the Human Head 
(Watson-Guptill, 1989), and Faigin’s 
The Artist’s Complete Guide to Facial 
Expression (Watson-Guptill, 1990) are 
two excellent references with more 
generalized information. 


| ees jury is still out on what is the 
most efficient method for creating 
animated facial expressions. While 
senerating morph targets can some- 
times be faster and often allows fine 
tuning of facial animation, taking the 
time to build a skeletal System with 
linked expressions can save time in 
the long run, especially when your 
game requires a large number of ani- 
mations. In any case, if you're sold on 
using morph targets as your end 
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target. The bottom line is that your 
technique will be determined by two 
factors: your engine’s animation SYS- 
tem, and your animator’s expertise in 
the given method. 


4 n order to set up our hierarchy cor- 
rectly, we need to identify the areas 
of the head and face that will be ani- 
mated. We get most of our informa- 
tion about a person’s mood by looking 
at two distinct regions of the face. The 
highlighted areas in Figure 2 enclose 
the upper and lower “active regions” 
involved in facial expression. This is 


where we will invest the bulk of our 
time and energy setting up the hierar- 
chy. The upper facial node acts as the 
parent to the nodes controlling the 
eyes, eyelids, and eyebrows. Similarly, 
the lower facial node is the parent for 
the nodes of the lower jaw, mouth, 
and tongue. Breaking down the hierar- 
chy in this way will do two things for 
uS. First, by compartmentalizing the 
areas of facial expression, we have bro- 
ken down the problem of facial anima- 
tion into smaller, more easily managed 
tasks. Second, by creating a preset list 
of animations for each region, we will 
be able to generate a widely varied set 
of facial expressions with relatively lit- 
tle effort. 


n order to create the motion of the 
i human face accurately, a basic 
understanding of the underlying mus- 
cle and bone structure is needed. Figure 
3 shows where each node corresponds 
to the mesh, while Figure 4 shows each 
node’s relative position in the hierar- 
chy. As you can see, with a few excep- 
tions, the nodes are bilateral, and lend 
themselves to being moved in pairs. 

A. THE SKULL. This will serve as the 
top node for the hierarchy. Several of 


Root Node 


Upper Facial Node 


Eyebrows 


Lower Facial Node = 


— Mouth Nodes 
Tongue Nodes 


FIGURE 2. Regions of the face and nodal hierarchy. 








ARTIST’S VIEW 


FIGURE 3. Nodal placement. 


the muscle groups involved in facial 
animation are anchored to this bone. 
Although this node does not animate 
and could be represented by any arbi- 
trary shape, it is useful to approximate 
the general shape of the skull when 
creating a hierarchy, as this serves as 
the foundation for and aids in the 
placement of subsequent nodes. 

2. UPPER EYELID NODE (LEVATOR PALPE- 


lid is kept open largely by the force of 
gravity. This node works in conjunc- 
tion with number 9 below. 

5. TONGUE nopes. The tongue is an 
optional part of the hierarchy, 
although if you want to use close-up 
shots of the characters, it is a definite 
necessity. The muscles of the tongue 
are particularly versatile, and are 
among the strongest in the body. For 


For any game based within a virtual 
environment, the player’s enjoyment is 
directly linked to how immersive that 


environment feels. 


BRAE). This raises the upper eyelid, as in 
a surprise or fear response. Because the 
upper eyelid has its own muscle group 
and the lower does not, most of the 
motion associated with the eyelids 
occurs in the upper eyelid. 

%. EvYENODE. The mesh of the eyeball 
should be separated from the rest of 
the face so that it can turn freely with 
the node. Also consider placing a con- 
straining expression on the eye nodes, 
so that they move in concert. 

4. Lower EYELID NODE. The lower eye- 


this reason, the full flexibility of this 
muscle should be represented by no 
fewer than three bones or nodes. 

6. THE JAWBONE. Pay particular atten- 
tion to the pivot point for this node. 
The jawbone is a hinged joint, with the 
pivot point located at the extreme rear 
point on the bone, where it is hinged 
to the skull. The tongue nodes and 
most of the lower mouth nodes are 
children of this node. 

7. THE FRONTALIS. This muscle is 
responsible for raising the eyebrows 





vertically. A bilateral sheet-like mus- 
cle, it is connected to the thick fiber of 
the scalp immediately beneath the 
hairline, and inserts into the skin 
directly under the eyebrows. Loe 
action of the frontalis is seen in such 
expressions as surprise, sadness, and 
fear, although it also sees action dur- 
ing regular conversation since raising 
the eyebrow is one of the most com- 
mon facial gestures. We often do this 
in concert with or in place of hand 
gestures during normal speech. 
Although this muscle can be repre- 
sented by a single node, many people 
have control over the individual sec- 
tions of the frontalis, enabling them 
to raise one eyebrow, while lowering 
the other (the “Spock” eyebrow). For 
this reason, there are two nodes, one 
for each side of the face. 

8. THe corruGator. This muscle 
group, also known as “the scowling 
muscle,” is actually comprised of two 
muscles, the corrugator and the pro- 
cerus. These muscles anchor to the 
skull at the top of the nasal cavity, and 
at the inside corners of either eye sock- 
et. The basic function of the corrugator 
is to pull the eyebrows down , while at 
the same time bringing them closer 
together. Used primarily in conjunc- 
tion with the frontalis, this muscle is 
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13 Mentalis, ete. 


FIGURE 4. Each node’s relative position in the hierarchy. 


associated with the acts of crying, 
scowling, and extreme concentration. 
D. ORBICULARIS OCULI. This is a large 
muscle group made up of concentric 
rings of muscle tissue, totally encircling 
the eye and extending into the cheek. 
As this muscle contracts, it tends to 
squeeze the eyes shut while at the same 
raising a good portion of the skin of the 
cheek. This muscle sees action whenev- 
er we squint, laugh, or smile, and is the 
primary operator in expressing pain. 
Although this muscle group can be 
approximated by two separate nodes (as 
in Figure 3), a single spherical node 
encompassing the eye can also be used. 
In that case, scaling the node down in 
the x, y, and z axis would approximate 
the contracting of the muscle group. 
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10. LEvAror LaBiI SuPERIORIS. Also 
called the “sneering muscle,” this mus- 
cle group uses a three-point anchorage 
spanning from the bottom edge of the 
eye socket to the lower ridge of the 
cheekbone. The muscles converge to a 
single point and insert into the skin 
just above the upper lip. Contracting 
this muscle tends to raise the upper lip 
towards the nostrils. In the real world, 
this muscle seldom sees action, except 
when expressing disgust, disdain, or 
loathing. We approximate it here by 
using a single node, which for us will 
be additionally useful in properly shap- 
ing the skin of the cheeks over several 
different expression groups. 

11. Orsicutaris oris. This is a banded 
group of muscles just under the surface 


of the upper lip, which is anchored at 
each end by the muscles at the corners 
of the mouth. Also called the “lip 
tightener,” this muscle works in con- 
junction with the levator labii superioris 
to create the expressions of disdain 
and loathing. It is also used for pursing 
the lips, as when someone experiences 
deep thought or concentration. 

12. ZyGomaricus MAJOR, RISORIUS/ 
PLATYSMA, AND THE TRIANGULARIS. These 
three muscle groups are responsible 
for pulling the corners of the mouth 
up, Out, and down, respectively. 
Approximated here by the action of a 
single node, these muscle groups are 
active over a range of expressions, 
from the exclamation of joy and plea- 
Sure, to the extreme stress of pain ora 
tragic loss. To effectively mimic the 
action of these muscles, it is necessary 
to incorporate most of the nodes in 
the lower region of the face. Test this 
out by smiling broadly or frowning 
severely, and you will see just how 
much real estate this muscle affects. 
This will also be one of the most 
active nodes, since for any speech or 
mouth movement, this node will be 
used to approximate the motion at 
the corners of the mouth. 

13. DEPRESSOR LABII INFERIORIS AND THE 
MENTALIS. [he combined action of these 
two muscles tends to pull the lower lip 
down (as during speech), or to push it 
upwards (giving a pouting expression), 
Here again, the Opposing motion of 
two separate muscle groups can be 
approximated by a single node. 


Now that we’ve identified the major 
facial muscle §roups and created our 
hierarchy, it’s time to apply the skeletal 
Structure to the mesh, and weight the 
facial vertices appropriately. The last 
Step will be to set up a lip-synching 
table and expression list, and from 
there we’ll be able to start animating. 
That’s where we'll pick up next month. 

For more information about facial 
animation from a programmer’s per- 
spective, please review Jeff Lander’s 
columns, “Read My Lips: Facial 
Animation Techniques” (Graphic 
Content, June 1999) and “Flex Your 
Facial Animation Muscles,” which 
appears in this month’s issue of Game 
Developer. @ 











ARTIST’S VIEW 





Technology Update: Dominatrix for Softimage 


n May’s column, “See Jane Walk,” we examined character 

animation techniques used in the industry, focusing partic- 

ular attention on motion capture. Until very recently, 

motion capture animation in Softimage was a painfully rig- 
orous exercise that limited animators to using a skeletal hierar- 
chy based on a nodal constraint system. In layman’s terms, the 
skeleton which the motion capture data fit onto was not neces- 
sarily the same type of skeleton which animators preferred to 
use while animating. The mo-cap skeleton had to match the data 
exactly, or the animation wouldn’t work. Subsequent data 
manipulation on a mo-cap skeleton often proved restrictive, so 
that mixing modes of classical animation with mo-cap was prob- 
lematic at best. Thanks to the team at House Of Moves, mo-cap in 
Softimage just got a whole lot easier. 

Taylor Wilson, the CTO of the Los Angeles-based motion cap- 
ture studio, has spearheaded a technique dubbed Dominatrix, 
which works to separate the motion data from the skeletal hier- 
archy; in essence, this technique frees up the animator to use 
whatever style of skeleton with which he or she feels most com- 
fortable. For example, say you want to use motion capture data 
for your character, but you’ve already started animating using 
another method. With the previous restrictions in Softimage (and 
most other animation tools), you would have to scrap all your 
previous work in favor of the mo-cap skeleton which matched the 
mo-cap data. Now, with Dominatrix, you can combine previously 
recorded motion capture data with other forms of character ani- 
mation without ever having to change your skeleton. 


he process is extremely simple. You merely provide 
House Of Moves with your character’s skeleton (sub- 
mitted in a Softimage .HRC file), and they do the rest. 
The motion capture data, originally stored in the 
Acclaim format, is washed through a set of proprietary tools 
which correct for any differences in proportion, orientation, and 
number of nodes between the standard Acclaim skeleton and your 
character’s skeleton. Then the data is mapped onto your charac- 
ter’s skeleton, and saved out as a Softimage .ANI file. Dominatrix 
can also generate IK constraint information to drive any of the 
limbs of your character. (This is in place of or in addition to using 
tradition positional/rotational keys at each joint.) The level of 
customization possible allows you to set up your character pretty 
much any way you want, with the simple restriction that the char- 
acter’s skeletal hierarchy has the same parenting relationships as 
the mo-cap skeleton. This is illustrated in Figure 1, which shows 
several humanoid skeletons all sharing the same hierarchical 
relationship (two arms, two legs, a torso, and so on). Note that 
the limbs of each skeleton are different in length (some have long 
arms, some short) and that the number of nodes in any given limb 
can differ between characters (the War Giant character, for exam- 
ple, has four nodes in his leg whereas the human character has 
only three). Dominatrix can accommodate all of these skeletons 
with the same set of motion capture data. 











Since this is something | had to see for myself, | opted for a 
test run of Dominatrix using DRAKAN’S main character, Rynn. Her 
Softimage skeleton is chock full of extra nodes in her chest, 
arms, and legs, and doesn’t match up very closely to the Acclaim 
skeleton. Within a few days of sending off the Softimage skele- 
ton to House of Moves, | received several martial arts animation 
files (see Figure 2) which had been mapped onto the character. 
When | imported these onto Rynn’s skeleton they worked flaw- 
lessly, and | have to say it was the least painful experience with 
motion capture I’ve ever had. The technique is called Dominatrix, 
and it’s available (at present, exclusively) from House Of Moves 
(http: / /www.moves.com). — Mel Guymon 
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FIGURE 1. Multiple Softimage skeletons. 


FIGURE 2. DRAKAN’s main character performing a martial 
arts move. 
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solves problems. Reduce your build time by as much as 90% with our 
industrial-strength software development tools. CodeWarrior tools are €asy to use and 
they support multiple platforms, so you save even more time - shorten the Learning 
curve, cut support costs, reduce Porting times. CodeWarrior decreases Production time 


and increases profit. No problem with that, right? 








Mpath, HearMe and Mplayer 


remember talking to an Electronic Arts executive at Intel’s Pentium II launch 
this past March. He was both roused by his company’s plans in the online 


gaming market, and very self-conscious. | wondered if he was nervous about 





the prospects, OF whether he was concerned that no one else would jump in 


and take a big bite of the pie. In April, realms occupied by the really big fish, they deployed with the help of PSINet 
when Mpath decided to go public, | such as Disney Online. in late 1995. In January 1999 the com- 
had a chance to think some more pany launched HearMe.com, its sec- 
about the issues facing the server side — | ah ers a a ond live community site. While most 
of the game business. There are some The Online Gaming Business Model game developers are familiar with 









sobering lessons to be learned from Mplayer.com, they may not know as 

Mpath, not only about online gaming, oe path was founded in January much about HearMe.com. 

but also about what it takes — beyond 1995. The company states as its HearMe.com currently consists of 

games — to make it in cyberspace. If business practice that it operates and seven live audio communities, making 

you look closely at Mpath’s strategy licenses live community Internet sites live audio interaction available to peo- 

you ll find that it could almost form to such companies as SegaSoft ple whose interests extend beyond 

the foundation of a big publisher, such Networks. Mplayer.com is the compa- entertainment. Mpath managed to 

as Electronic Arts, going wider thanits ny’s low-latency gaming platform, oF raise approximately $34.9 million 

core demographic, perhaps into the live community as it is called, that between its inception and the end of 

1998 through the sale of equity securi- 

FIGURE 1. Mpath’s statement of operations data for the fiscal years indicated as ties to CSK, SegaSoft’s parent Compe 


a percentage of total revenues. n/m means “not material.” ny, and various venture capital (VC) 
and strategic partners. In January 


1999 the company raised an addition- 
al $20 million through VCs to prepare 
for going public. That’s a big chunk of 
change, but Mpath hoped to raise over 
$68 million from its initial public 
offering (IPO). 

Being an online contender requires a 
big investment, even in the gaming 
niche that Mpath has carved for itself, 
but their long term strategy seems to 
indicate much more. The company 
derives its revenues from two business 
units, Live Communities and Mpath 
Foundation. Live Communities generate 
advertising revenues, and this includes 
advertisements targeting the key demo- 
graphics at Mplayer.com and 
HearMe.com. Mplayer.com consumers 
are predominantly male. In sports and 
game player communities, more than 
90 percent of the participants are male, 
and the people within those communi- 
ties are typically between the ages of 13 
and 50. However, approximately 40 pert- 
cent of the participants in classic games 
and the casino community are women. 
The people within these communities 
are typically between the ages of 25 and 
50). As for HearMe.com, currently more 
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than 40 percent of the participants are 
women, and its members are typically 
between the ages of 13 and bs 

Mpath Foundation is the licensing 
and services arm of the company. In 
1998, CSK Sega, Sony and Electronic 
Arts accounted for 23, 12 and 10 per- 
cent of total revenues respectively for 
Mpath Foundation. In 1997, CSK Sega 
and Sony accounted for 35 and 11 per- 
cent of total revenues respectively. 
Combine both Live Communities and 
Mpath Foundation, and the business 
model for Mpath is simple. The compa- 
ny created Mplayer.com, and now 
HearMe.com, to showcase the applica- 
tion of its technology, and to put upa 
barrier to market entry for other parties 
interested in competing with so-called 
live communities. There’s a growing 
mesh of third parties that tie in with 
Mpath’s live communities, whether 
they be advertisers, vendors who have 
products that sell directly through 
Mpath operations, or Magazines and 
portals that link to Mpath’s demo- 
graphics. Mpath Foundation then takes 
the technologies that make all this 
happen, and applies them to other 


ey ee Oe | Rahmat 


places on the web. POP.X is the prod- 
uct that Mpath Foundation Sells, a 
toolkit for enabling live communities. 
And here is where it gets interesting, 
and where the lessons emerge. 

Mpath is, like most Internet compa- 
nies, primarily interested in growth. 
This means pushing up memberships 
and subscriptions. In order to do that, 
the company needs content from third 
parties, and that makes for more enter- 
taining reasons to build an audience. 
Furthermore, Mpath is also putting a 
lot of money into research and devel- 
opment of key technologies that wil] 
better manage its communities, and 
Streamline the low-latency web enter- 
tainment experience. 


Growth Maxims 


path Foundation consists of a 

gtowing list of online entertain- 
ment Companies, including CSK Sega, 
Electronic Arts, Fujitsu, GIECH, and 
LG Internet. The Mplayer.com service 
Now comprises three active communi- 
ties built around common interests, 


and offers more than 100 of the most 
popular online multiplayer games. 
According to Mpath’s internal com pa- 
ny reports, total usage time on 
Mplayer.com exceeds 200 million user- 
minutes per month as of January 1999, 
compared to 67 million user-minutes 
per month back in January 1998. 
Mplayer.com has become the tenth 
largest Internet site in terms of total 
usage time per month, according to the 
company’s own usage data. 

The opportunities for Mpath are 
extensive. The technology, the fact 
that they are growing beyond online 
gaming (a feat that some of their more 
esoteric competitors have failed to 
negotiate), and the general branding of 
the company’s products all have 
helped the company achieve a leader- 
ship role. However, with almost two- 
thirds of their revenue coming from 
their technology licensing and SEIVICes, 
and advertising and subscriptions 
being very speculative means of mak- 
ing a profit on the Internet, Mpath 
faces some challenges. 

First of all, there is the ULTIMA 
ONLINE franchise model. It’s worked for 
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Electronic Arts, and it means that pub- 
lishers are willing to risk some effort 
and cash in order to create their own 


ers at heart means that these publishers 
will probably want to build the server 
technologies themselves, and tailor 


tion. These technology barriers are 
becoming increasingly difficult to main- 
tain, relying as they do on similar objec- 












tives having to do with managing and 
maintaining online communities. 
Everyone wants to trap the consumer 
on their site and keep them there. As for 
spending lots of money to get traffic, 
that depends on content. Otherwise, 
your customer has to do very little in 
order to go somewhere else. Point, click, 
and carriage return is about all it takes. 
And that’s the best news that smaller 
developers have heard in a long time. 
Building communities around content, 
developing a franchise, and creating 
multiple revenue streams are all within 
the reach of any Internet Savvy develop- 


them to specific gaming experiences 
and genres. In other 
words, the one-size-fits- 
all approach may not be 
the ideal way to go for- 
ward. Of course, Mpath 
also makes a good acqui- 
sition target for a com- 
pany that wants to get a 
head start, and despite a 
very successful IPO, the 
volatility of Internet 
stocks could have an 
impact on the future 
growth of the company. 


live communities. Being game develop- 


FIGURE 2. Statement of operations (in thousands of 
dollars) from January 9, 1995, to year ending 
December 31, 1998. 
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Using the Java Native 
Interface within Games 


Sai 
BY BERND KREIMEIER 
eeleineeiaeemememe aaa a, 


ou have heard of Java. Actually, you probably have had a hard time trying to escape the 





hype surrounding it since 1995. 


j 


There are, however, compelling reasons to use Java in 








shrink-wrapped games. Two Major options that have recently been 





examined by professional game developers are using Java as your 
scripting language, and using Java for safe run-time downloadable code 
that gets executed on the client. In both cases, it is the interface to Native 
code that you end up dealing with the same glue that is required to make Java 
work on your PC in the first place. If using a portable, standard, object-oriented 
programming language with built-in security within your game sounds appealing, 
then you should become acquainted with the Java Native Interface (JNI), which is your 


tool to write “dirty” Java — Java code that is tightly integrated with your native code. 


tract work on Java for games, and worki 


bk@gamers.org. 

















There is an abundance of information 
about “pure” Java, and this is not the 
place to explain all the actual and 
alleged advantages of Java technology. 
Unfortunately, many of the highly 
touted “pure” solutions quietly omit 
the sophisticated native machinery at 
work under the hood. Instead of look- 
ing at game applets running in web 
browsers, this article sizes up possible 
real-world uses for Java, and looks at 
the ways some game developers are 
already using Java for their titles. 


A Brief Recap of Virtual Machines 


& riefly, here are the key compo- 
nents that are relevant to this dis- 
cussion of Java: 

JAVA BYTECODE. [his is pseudocode for a 
stack-based processor described in the 
Java specification. Valid bytecode has to 
satisfy a lot of requirements, but if you 
really wanted to, you could actually 
write it by hand with a decent hex edi- 
tor and the specifications. 

THE JAVA VIRTUAL MACHINE (JVM). Java CPUs 
have not yet conquered the market, so 
software must translate Java instructions 
into the language the PC hardware can 
understand. The JVM is a multithreaded 
program for your operating system, sup- 
plied from various vendors, that exe- 
cutes Java bytecode and maps bytecode 
to native instructions. The Java specifi- 
cation places few restrictions on the 
actual implementation of a JVM. You 
can find Open Source JVMs on the 
Internet, or even write your OW!) clean- 
room implementation. You can also get 
the sources of Sun’s reference Java 
Development Kit (JDK) and negotiate a 
license for commercial use. 

THE JAVA PROGRAMMING LANGUAGE. This is an 
object-oriented language, designed 
with a subset of C++ in mind. It has 
run-time bounds checking, it is restrict- 
ed in terms of memory access and han- 
dling, and it has simplified inheritance 
patterns. The language features an 
overwhelming inflation of extension 
APIs and core classes. 

It is important to understand that 
these three components are entirely 
separate. A JVM will happily execute 
valid bytecode that was not generated 
from Java source — for instance, from a 
compiler that maps a language like 
Scheme to bytecode. You could also 
implement a simplified virtual machine 


avenue of hand-tuned virtual assem- 
bly” (as John Carmack of id Software 
referred to virtual CPUs) to using a JVM 
without ever coding in Java (converters 
mapping C or C++ subsets to Java are 
feasible), all your tampering needs can 
somehow be addressed. 

A lot of confusion originates from dis- 


that skips validation and executes byte- 
code no compliant JVM would accept, 
or create a virtual machine that has no 
garbage collector thread at all. In addi- 
tion, there are compilers that generate 
native code from Java source files — 
you don’t need a JVM to write an appli- 
cation in Java. From the “amusing 






FIGURE 1. JN/ built-in data types. 





JNILTRUE / JNLFALSE (eid: 














~ boolean jboolean 
byte a ibyte , signed 8 bits 
short jshort signed 16 bits 
int ‘ae jint — signed 32 bits err | ” 
long , yi jlong signed : 0 ice - oe ’ 
float jfloat , IEEE754 32 bits _— E 
"double idouble oa ‘ 
char jchar Miatcode™ a3 bits siento ai 
void void. N/A " ~ 3 deren V 


* The char data type is the only unsigned integral type available in Java. It is interpreted 
as 16bit Unicode, and mapped from/to UTF-8 encoding in |/O translations, but it is 

expanded to int for arithmetic operations, and all integral arithmetic operations 
_ are available. | = 








FIGURE 2. JNI visible class tree. 


























java.lang.Object ‘Sbject <5 Ljava/lang/Object 
___ java.lang.Class eka jclass Ljava/lang/Class 
_ java.lang.String jstring Ljava/lang/String 
___ java.lang.Throwable_ jthrowable Ljava/lang/Throwable 
Pcl erie jarray N/A* 
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* There is no java.lang.Array class, So JNI's jarray has no real equivalent in Java, just like jmethodlD, 
jfieldID or jvalue. 






+ There is no jstringArray, So arrays of Strings map to the jobjectArray handle. So do arrays of Class, 
Throwable and any other subclass of Object, as well as mixed arrays and arrays of arrays. 
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cussions that fail to distinguish the vari- 
ous components of Java technology. All 
of these are required parts of any JDK, 
however. The only JDK that is actually 
called JDK is the reference implementa- 
tion provided by Sun Microsystems for 
Solaris and Win32, and its licensed 
ports to other platforms. 

The concept of a virtual machine for 
game scripting is not new, and not 
restricted to Java. John Carmack 
recently decided to use a custom virtu- 
al machine and bytecode created by a 
modified C compiler (LCC by Fraser 
and Hanson) for id’s upcoming game, 
QUAKE 3: ARENA. Carmack once pointed 
out that game coders “have more 
urgent things to do than design lan- 
guages.” Ironically, he is now engaged 
in designing his own virtual machine 
and native interface. Technologies like 
Java’s just-in-time compilation and 
HotSpot optimization originate In the 
Java technology mainstream, and they 
are powered by more resources than a 
single game Company could ever com- 
mand. If you can make Java work for 


1 | f. 
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your game, then you will benefit from 
this momentum. 


Talking to the Natives 


S ame coders usually do not trust 
cross-platform APIs based on lay- 
ers of abstraction. Interpreted bytecode 
does not typically appeal to an indus- 
try that still counts on assembly to get 
a performance edge. Windows-based 
games sell, period, and portability is 
not really an issue. 

Compare this to the holy grail of “100 
percent pure Java.” Mainstream Java 
technology is seemingly meant for tiny 
“samelets,” not serious games. Besides, 
despite the bloat, there are bits and 
pieces missing from the Java core class- 
es: access to certain devices and system 
services is simply not available (and 
might never be, for design and security 
reasons), and politics sometimes gets in 
the way (witness the lack of Java 
OpenGL bindings). However, if you take 
a closer look, it turns out that there is 


Native code making use of Java. 


// Java class, server-side Game Logic scripting. 


package somegame; 

class ScriptEngine { 
// This uses native error() callback. 
public static void init() 4} 


public static native void error( int code ); 


} 

#include <jni.h> 

// Native error callback provided to Java. 
extern void SV_ErrorScriptEngine( jint i ); 


JNINativeMethod svSE = {"Java_somegame_ScriptEngine_error","(1)V", SV_ErrorScriptEngine }; 


// Native function to set up scripting module. 


void SV_InitScriptEngine( JNIEnv* env ) { 
jclass clazz = NULL; 
jmethodID method = NULL; 
jint err; 


// Lookup class, loads the class if not yet done. 
clazz = (*env)->FindClass( env, "somegame.ScriptEngine® y 


if (-clazz.J= MLL Jt 


// Register native method, returns zero on success. 
err = RegisterNatives( env, clazz, &svSE, 1); 


// Lookup Java method1D. 


method = (xenv)->GetStaticMethodID(env, clazz, ‘idge’. "OF"; 


} 


// Handle errors. 


// Call static method. 


(*env)->CallStaticVoidMethod (env ,clazz,method) ; 


} 
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always native code at the very heart of 
all that “pure Java”: there is a JVM writ- 
ten in native code, and core classes part- 
ly implemented as native code. Here 
reigns the Java Native Interface, gluing 
together Java and native C/C++ code, 
and it is the key to combining your 
native code with Java technology. 

JNI is part of the Java specification, 
and it’s a mandatory part of all Java 
implementations. Sun was recently 
granted a court injunction forcing 
Microsoft to add JNI to the Microsoft 
Java implementation. Ideally, .DLLs 
and binaries using JNI should be byte- 
compatible for a given platform. The 
1997 JNI specification is available 
online, and there are also books on the 
subject, so this article includes only a 
brief summary of it before we get into 
its applicability in games. 

The first task JNI must solve is getting 
the JVM and user-written native code to 
agree on built-in types and memory lay- 
out to exchange data (see Figure 1). 
Only some of the core classes are repre- 
sented for the native code (see Figure 2) 
— most of them arrays of the built-in 
types. 

What about jclass and jobject? JNI will 
not hand you the memory layout ofa 
Java object, but it must provide you a 
handle. It even preserves the relation- 
ship between java.lang.Object and 
java.lang.Class. A jclass object can be cast 
to jobject safely in any JNI that complies 
to the Java specification (non-compliant 
implementations have been found). JNI 
is foremost aimed at C (the C++ bind- 
ings are just inlined wrappers), and no 
support for object-oriented program- 
ming on the native side is offered. With 
the exception of Throwable, String, and 
arrays, all classes have to be squeezed 
through the jobject and jclass represen- 
tation. Arrays of arbitrary classes 
(including String) will always be mapped 
to jobjectArray. JNI defines JNI_FALSE/TRUE, 
a jvalue union type, and a jsize for your 
convenience. 

Further, handles have access to fields, 
and also call methods of classes and 
objects. It might look like java.lang.~ 
reflect.Field and java.lang.reflect.Method 
are the Java equivalents to JNI’s jfield1D 
and jmethodID, but JNI predated the 
Reflection API, and actual reflection 
support was added only to the latest JNI 
revision. 

Caching field and method IDs Is a 
good idea, as retrieval involves a string 
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lookup. Be warned that caching can get 
tricky in applications with multiple 
threads and class loaders. You will have 
to keep an eye on the garbage collector 
as well — without a strong reference 
acquired by NewGlobalRef(), the garbage 
collector might remove the object your 
native code is still referring to. 
Likewise, dangling references not 
removed by DeleteGlobalRef() can keep 
obsolete Java objects from being col- 


LISTING 2. 





package somegame; 
class GameMain { 


lected. Use DeleteLocalRef() to avoid 
accumulating temporary references 
within loops. JDK 1.2 offers limited 
support for weak references, too. 

Within your native code, all will 
revolve around the JNIEnv interface 
function table — your door to the Java 
side. It provides methods to handle ref- 
erences, create objects, load classes, 
access fields and call methods. Further, 
you get utility functions to iterate 


Passing Arguments from Java to C. 





// Minimal Java control code wraps legacy engine. 


protected static native int nativeMain( String[] args ); 


public static void main( String[] args ) { 
int ret = nativeMain( args ); 


} 
static { System.loadLibrary ("Game"); } 
} 


// Handing over the arguments to native code. 


// This code will be put into the Game.DLL. 
#include <jni.h> 


extern int gameMain( int argc, char#+ argv ); 


JNIEXPORT jint JNICALL 


Java_somegame_GameMain_nativeMain ( JNIEnv« env, jclass cls, jobjectArray jargv ) { 


jint res; 

jint argc; 

JANE Ly 

jboolean isCopy; 

Jjstring jstr; 

jsize len; 

const char* cstr; 

Jargc = env->GetArrayLength(jargv) ; 
for (i= 0; isarge; i++ ) { 


jstr = (jstring) (+env)->GetOb jectArrayElement (env, jargv, i); 
cstr = (const char*) ((env)->GetStringUTFChars( env, Jstr, &isCopy )); 
// We copy - we have to release, and we don’t want to accumulate local references. 


argv[i] = (char*)malloc( strlen(cstr)+1 ): 


strcpy( argv[i], (const char*)cstr ) 
// Did the JVM copy as well? 
if ( isCopy == JNITRUE ) { 


(+env)->ReLeaseStringUTFChars ( env, jstr, cstr ); 


} 
// Clear local reference. 
(+env)->DeleteLocalRef (env, jstr):; 
} 
// Call our main() now. 
res = (jint)gameMain( (int)argc, argv ); 
// Release allocated memory . 
for ( i=0; i<arge; i++ ) { 
free( argv[i] ); 


// Return to Java. 
return res; 


GAME DEVELOPER JULY 1999 


arrays, throw exceptions, and perform 
monitoring to make the native code 
threadsafe. Finally, an executable can 
also register functions as native meth- 
ods, making code known to the JVM 
without dynamic linkage. 

Listing 1 is a small example showing 
how native code can use Java to get 
things done, and how a native callback 
is registered with the JVM. Listing 3, 
discussed later on, does the Opposite: it 
shows how Java calls native code. 

It is tempting to use static (class) 
methods, as you do not have to handle 
an object in addition to the class. In 
many Cases this is absolutely sufficient 
— servers will likely not run two script 
engines in parallel. In many other cases 
though, this leads to bad object-orient- 
ed design on the Java side. 


Double Indirection: The Catch-22 


T here have been competing native 
interface APIs proposed, most 
notably Microsoft’s Raw Native 
Interface (RNI). The problem with RNI is 
that it exposes the underlying operating 
system and JVM implementation, 
which makes it impossible to port to 
another VM. 

In some ways, the problem with JNI is 
that it does not expose the VM imple- 
mentation. JNI makes you go through 
pains to ensure that native code never 
gets to see how Java objects are laid out 
in memory. Consequently, the native 
code has to deal with indirections every 
step along the way, many of them ullti- 
mately leading to table and string 
lookups. This is not good for application 
performance. See the example in Listing 
2, where we pass command line argu- 
ments from Java to C, which involves 
references, arrays, and conversion to 
UTF-8 (the canonical two-byte Unicode 
encoding used by Java). 

Tools such as javah generate C header 
files containing proper function proto- 
types (name and signatures) from a Java 
Class. These tools are already Unicode- 
aware, thus using underscores and 
other special characters in Java method 
and class names can lead to surprising 
results. The code in GAME.DLL will be 
linked to the class by the JVM by call- 
ing java.lang.System.loadLibrary ("Game") 
automatically. 

The example in Listing 2 implements 
a minimal Java wrapper around native 
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legacy code. Given all the implicit and 
explicit copying, we somehow seem to 
have come full circle: to get rid off some 
portability-related Java overhead, we 
decided to use native code, only to find 
out that the JNI design hampers the 
interaction between Java and native 
code to ensure portability. Now what? 
Well, there are basically two ways left to 
increase performance: 

4. Brute force. You could switch 
tools and compile to native code. If 
you pursue this option, make sure 
your Java compiler supports JNi as 
well, and that it doesn’t just compile 
pure bytecode. 

2. Smart design. You could accept 
the limitations of the JNI, and design 
your native and Java modules in a 
way that streamlines the interface 
between them. 

Mind you, your native code by itself 





will be as fast as it gets. It’s only the 
transfer of parameters and results back 
and forth that, inside an inner loop, in- 
curs significant performance penalties. 


The Invocation API 


ee ow that you have seen some 
means to glue Java and native 
code together, where does a game 
developer actually get access to the 
virtual machine? The common answer 
is the Invocation API. The Invocation 
API allows you to embed the JVM into 
your native applications. It provides 
the means by which you can retrieve 
an existing JVM attached to your 
application, or launch one with prop- 
er configuration settings. Listing 3 
shows how to invoke the JVM in an 
application, using JDK 1.2. You can 


3 How to invoke the JVM in an application using JDK 1.2. 


// JDK 1.2 Invocation example 
#include <jni.h> 
// Setting some standard options. 


extern jint JNICALL Printf( FILE +f, const char *fmt, va_list args ); 


extern void JNICALL Exit( jint code ); 
extern void JNICALL Abort( void ); 
tdefine NOPTIONS = 5; 

JavaVMOption OPTIONS[5] = 

{ 


{ "classpath", (void) "C:\\ java\Lib\classes. zip; D:\Game\classes" }, 


{ "verbose", (void*)"jni,gc" }, 
{ "vfprintf", (void*)Printf }, 
{ ‘age. (voidt)Exit }, 
£ "abort, (voids) Abort } 

s 


// Create a JDK 1.2 JavaVM as desired. 


JNIEnv* SV_InitJavaVM( JavaVMOption* options, jint nOptions ){ 


JavaVMInitArgs vm_args, 


JavaVM*# | vm_handle; // not preserved 
JNIEnv* = env; // return to caller 
jint ret; 


// Request version 1.2 
vm_args.version = JNI_VERSION_1_2; 


ret = JNI_GetDefaultJavaVMInitArgs(&vm_args) ; 


if ( ret==0 ) f 
vm_args.options = options; 
vm_args.nOptions = nOptions; 
vm_args.ignoreUnrecognized = JNI_TRUE; 


ret = JNI_CreateJavaVM( &vm_handle, &env, &vm_args ip 


if ( ret==0 ) { 
return env; 
} 
} 
// error handling 
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use code like that shown in Listing 1 
to get Java classes loaded and execut- 
ed. If you do not want to encapsulate 
native method code into .DLLs, then 
your application can use the JNI func- 
tion RegisterNatives() to make native 
functions from the executable known 
to the JVM. That way your game 
would ship without any .DLL. 

Most JVM and compiler implemen- 
tations fully support JNI, as it is need- 
ed to handle cleanly implemented 
core classes. (Interestingly, even Suri’s 
own JDK does not always use JNI 
internally.) But the Invocation AP! 
was sometimes omitted from JDK 
ports and third-party JVMs. If you 
want to use the Invocation API, make 
sure your tools and targets support it. 

Worse still, sorme JDK ports support 
Invocation, but do not do so properly 
(including some revisions of Sun’s own 
Solaris reference implementation). 
Invocation requires threadsafe .DLL 
handling, which is not always granted 
(for instance, in some Solaris and Linux 
revisions). If the dynamic linking is not 
threadsafe, your application will suffer 
spurious errors during startup. 
Furthermore, the official Java specifica- 
tion now sanctions limitations of the 
reference JDK that affects DestroyJavaVM(). 
It is not possible to destroy a JDK JVM. 
Consequently, you can’t invoke anoth- 
er one from the same application — 
multiple JVMs, whether subsequent or 
in parallel, are not possible. Once you 
lose your JVM for whatever reason, 
your application must terminate. Fortu- 
nately, most of the other pitfalls were 
smoothed out last year. 

In Listing 3, ignore the JVM handle, 
which you can always retrieve by call- 
ing the JNI function GetJavaVM(). A more 
flexible approach to invoke the JVM 
calls GetCreatedJavaVMs() first to check 
whether a JVM already exists, and uses 
AttachCurrentThread() to make itself 
known if one is found. The question 
is, do you really have to invoke the 
JVM this way? 


Two Architectures: Embedded Java 
and Encapsulated Native Code 


ded ow you choose to obtain the J VM 
for your game is, in all likelihood, 
the most important decision you have 
to make when rigging up a Java-based 
game project. To a C coder, invocation 
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might seem a natural choice. This 
architecture is known as embedded 
Java. Your application is linked to a 
-DLL that provides a JVM, which hap- 
pily lives and dies within your applica- 
tion, completely at your disposal. It 
looks like this: 

// Engine piggy-backed with JVM or 

// Engine retrofitted with JVM 

// Set options, possibly parsing commandline. 
nUpt = SV_GetOptions (options, argc, argv): 

// Invoke JVM, get script engine started. 
SV_StartVM( SV_InitJavaVM (options, nOpt)); 
// Start the actual game. 

return gameMain( argc, argv ); 

On the other hand, if you write a 
pure Java game, or use the 
somegame.GameMain Class shown in Listing 
2, then some other application loads 
the JVM and hands it the Java bytecode 
of your game. This scenario is used 
when a web browser runs “gamelets”, 
for example, or when JDK’s java loads 
an executable JAR file. Whether your 
same uses native code or not, you do 
not have to concern yourself with 
invocation if the main loop is written 
in Java. Native method code will be 
encapsulated in Java classes, as long as 
the .DLLs required are loaded in time. 
It does not look like much of a differ- 
ence, but choosing one or the other 
might have a huge impact on your 
project. 


Embedded Java: A Natural Choice? 


L et’s look at an example that I call the 
“QUAKE 3 scenario.” Your team has 
nearly finished a game engine written in 
C or C++. The game has a large and sta- 
ble legacy code base that you don’t want 
to tamper with, yet there is a clear-cut 
need that Java might address, such as a 
new server-side scripting language, or 
support for client-downloadable code. In 
short, you want to retrofit an existing 
application with a Java component. 

The history of the QUAKE engine is a 
great example. QUAKE featured a custom 
scripting language (QuakeC), QUAKE 2 
introduced a server-side .DLL 
(GAME.DLL), and some QUAKE engine 
offspring now deploy client-side .DLLs. 

Embedding is possibly the best 
answer in all cases where you have to 
deal with C legacy code that is not 
implemented in an object-oriented fash- 
ion. The JVM is just another device that 
is initialized, configured, started and 
GAME DEVELOPER 
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Java wraps native code in PRAX WAR 


ritten by Billy Zelsnack 
during his days at 
Rebel Boat Rocker, 
PRAX WAR was destined 
to be the first major game to use Java for 
most of its code. With the exception of aC 
renderer, the PRAX WAR engine was written 
entirely in Java. Zelsnack explained the 
game this way: “We use JDK and JNI. The 
game itself starts from Java. | use Java as 
controller code for C. Java is very good at 
calling C code, but [it is] not necessarily as 
clean the other way around.” The design 
kept raw data (such as textures and 
sounds) on the native side, but made 
them accessible to Java as needed. Billy 
Zelsnack found few problems with the 
core classes (at one point, UDP network- 
ing performance was an issue), and found 
no problems with the most feared Java 
component, the garbage collector. The 
engine used just two threads — one 
thread that listened for incoming packets, 
plus the main loop itself. 

Unfortunately, Rebel Boat Rocker’s 
publisher, Electronic Arts, decided earlier 





shut down again. There are some restric- 
tions (for instance, you cannot restart 
the JVM once it has shut down), but in 
general, all you do is provide raw data 
(bytecode) to the embedded JVM much 
the same way you’d feed .WAV files to a 
sound device. If you do not want to use 
-DLLs at all, embedding is your solution. 
You also get a lot more control over the 


JVM that is used by your game. Ship- 


ping a Java Runtime Environment (JRE) 
with an embedded solution might save 
you support and maintenance 
headaches. It might also address some 
reverse engineering, tampering and 
cheating issues. 

If embedded Java is used, either C 
control code executes Java methods on 
the JVM which return the data, or the 
Java code in turn calls native methods 
to write back. You could have Java 
threads run in parallel to your applica- 
tion, but debugging an application that 
moves back and forth between native 
and Java execution stack frames can bea 
challenge to you and your tools — mul- 
tiple threads will make it even tougher. 

What problems are specific to using 
an embedded JVM? Some have already 
been mentioned, such as negligence or 
outright omission of the Invocation AP] 





this year to cancel the project, stating that 
the game had “missed its technology win- 
dow”. If Electronic Arts had had the kind 
of faith in Rebel Boat Rocker that Sierra 
has shown in Valve Software, we might 
have found that their assessment was 
straight to the point — maybe Prax War 
missed its technology window by being 
too early. 

Zelsnack summed up his Java experi- 
ence up this way: “Java opened up possi- 
bilities for the product that could not have 
been realized without its power. It was 
one of the things | was most excited about 
and proud of.” 


from some Java implementations, and 
potential problems you might face 
when falling back on compiling Java to 
native code. All of these problems can 
be overcome one way or the other, how- 
ever. The real danger might be much 
more subtle. 

Your legacy code has a certain design 
— possibly not object oriented at all if 
you used C, or possibly an object-orient- 
ed design that maps badly to Java if you 
used C++ excessively (if you made use of 
templates and/or multiple inheritance). 
In these cases, taking a single compo- 
nent of your game (for instance, the 
server-side game logic) and converting it 
to Java could introduce bugs and errors 
in formerly stable and tested code. 
Worse still, through JNI the design used 
in native code will proliferate into Java 
code, resulting in badly designed Java 
code. For example, if you never handle 
objects (see Listing 1 and its use of class 
methods), it is unlikely that you are 
using an object-oriented design. Legacy 
code tends to share memory using 
pointers for speed and convenience, 
which is not possible with JNI. You have 
to think hard and make judicious cuts 
to get a lean interface between Java and 
native code. High levels of abstractions 
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implemented as abstract base classes 
and interfaces usually work best: the 
more details you hide, the better JNI 
will work for you. 

Consider handling structured data on 
both sides of JNI, such as that used for 





LISTING 4. Java Proxy for a C++ object. 


package jni; 
class Proxy { 
/** Handle to retrieve C++ object, native side. */ 
private final int handle; 
/*x* Native LUT/constructor. */ 
private static final native int newNative() ; 
/xx Constructor, gets/creates a handle to a native object. */ 
public Proxy() { 
handle = newNative(); 
} 
/*x* Simplified fieldID. */ 
private final int SOME_FIELD = 1; 
/*x Accessor hiding the simplified retrieval. */ 
public final float getSomeField() 1 
return getFloat( handle, SOME_FIELD ip 
} 
/** Method that saves us many retrieval() functions. */ 
private final native float getFloat( int handle, int field ); 
} 
// Minimal C++ object, and JNI glue. 
class NativeObject { 
public: NativeDbject( jobject owner ){ 
this.owner = owner, 
} 
public: jobject getOwner() {return owner; } 
public: float someField; 
public: jobject owner; 
}; 
#include <jni.h> 
extern jclass InvalidProxyOwnerException; 
extern jclass InvalidFieldIndexException; 
// extern "C" implied 
JNIEXPORT jint JNICALL Java_jni_Proxy_getFloat 
( JNIEnv* = env, 
jobject owner) 
{ 
return (jint) (new NativeOb ject (owner) ); 
} 
JNIEXPORT jfloat JNICALL Java_jni_Proxy_getFloat 
( JNIEnv* env, 
jobject owner, 
jint handle, 
jint field ) 
{ 
// Truly dirty. Trust on blank finals. 
NativeObject* obj = (NativeOb ject*)handle; 
if ( obj->getOwner() != owner ) 
(xenv)->ThrowNew(env, InvalidProxyOwnerException, "access attempted by non-owner") ; 
switch( field ) { 
// Enums to be kept in sync manually... 
case 1: return obj->someField; 
default: 
(xenv)->ThrowNew (env, InvalidFieldIndexException, 
"access attempted by non-existing field"); 
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collision handling. Collision response is 
part of the game logic (does the player 
take damage, bounce, or die?) and is thus 
handled in the Java code in our example. 
Collision detection might be performed 
within the scene representation that is 


also used by the rendering code — 
almost surely native code you want to 
keep. In this scenario, your Java game 
logic might call native code to trace an 
object’s movement through the scene. 
This is where the level of abstraction is 

relevant. Take the QUAKE 2 representa- 
tion of a vector in 3D space: float[3]. In 


Java, this is best represented as a class 


with float x,y,z fields. This avoids array 
bounds checking overhead, and frees us 
from worrying whether JNI pins or 
copies the array. For objects that small 
and likely to have all their fields accessed, 
the simplest way to pass them back and 
forth is to unfold them on the stack as 
primitive data types, much the same they 
would be flattened for serialization. 

This solution is more the exception 
than the rule, however. In general, it 
pays off to hide as much detail as possi- 
ble on both sides. The game logic does 
not need to know whether axis-aligned 
bounding boxes or spheres are used for 
collision detection, it only has to initi- 
ate updates to position and size. For the 
actual trace in native code, it is irrele- 
vant whether a given entity within the 
bounding volume is a player, a mon- 
ster, or a fireball. 

Using a high level of abstraction on 
the Java side by sticking to abstract base 
classes and interfaces makes retrieving 
and caching method IDs in your native 
code much easier, since all objects with- 
in an inheritance tree will share the 
same signatures. You might find it safer 
to cache field and method IDs in class 
descriptor structs or C++ objects. Field 
access is more efficient, but exposes the 
internal implementation of your Java 
objects. JNI methods like GetFieldID and 
GetFloatField can be used instead of, say, 
GetMethodID and CallFloatMethod to access 
instance fields directly. 

You pass references as jobject handles 
instead of pointers to make Java data 
accessible to native code. The reverse is 
not possible: neither C structs nor C++ 
objects are visible to Java. You can 
address C++ objects or C structs with jint 
handles on the Java side, using more 
(hash table look-up) or less (typecast) 
safe ways to retrieve the effective 
address. A proxy class would then wrap 
native methods with public accessors, 
like that sketched out in Listing 4. 

Of course, if you want to avoid 
switch statements in the native 
method, or you want to wrap C++ 
accessor methods instead of exposing 
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fields, you could also implement the 
public Java accessor as a native method. 
Incidentally, maintaining the same set 
of enums in Java and native code is one of 
the problems that does not yet seem to 
have an elegant solution. If you are 
using a look-up table to retrieve point- 
ers for handles, the jobject argument 
might already be sufficient. 

A native proxy implemented as a C++ 
object or C struct could cache a global 
jobject reference along with method IDs 
and field IDs. Caching actual game 
data inside native code means that 
your proxies have to be kept synchro- 
nized with the master objects, or you 
will end up with consistency errors 
that are very difficult to track down. 

Alternatively, you could encapsulate 
the results or take a snapshot of a 
native object’s state in a new Java 
object created in native code, using the 
JNI function New0bject() to call a Java 
constructor. This approach works even 
better if your native and Java modules 
communicate by passing event descrip- 
tor objects to a queue. 


Encapsulated Native: A Magic Bullet? 


f you go down the road of Java con- 

trol code, be prepared to throw out 
legacy code whenever necessary. 
Encapsulation means dividing and split- 
ting your code base into tiny pieces — 
heaven if you are at liberty to design 
from scratch, hell if you have to handle 
code that is just sticking together. If you 
can't isolate native code modules and 
Wrap them with Java classes, then there 
won't be a secure migration path. 
Handling a native legacy code base 
might well take more time than gutting 
it and starting from scratch. If you con- 
sider abandoning C/C++ as your main 
language, an encapsulation architecture 
is definitely the way to go. 

Java and JNI will always have some 
performance disadvantages that make 
them unsuitable for time-critical inner 
loops. However, it can be very efficient 
write your control code in Java (which 
can account for up to 90 percent of a 
game’s total code base) even though that 
often takes up less than 10 percent of 
the overall processing time — especially 
for games that make the CPU spend half 
of the time in an OpenGL driver. This 
was the reasoning behind the PRAX WAR 
architecture that Billy Zelsnack imple- 
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mented at the now defunct game devel- 
opment studio, Rebel Boat Rocker (see 
sidebar, “Java wraps native code in PRAX 
War”). There are only a few areas in 
which native code is really needed, such 
as Managing raw data (textures and 





sound resources, for example), and ren- 
dering. Collision detection might best be 
done in native code shared with the ren- 
derer. Collision response, however, is a 
natural part of high-level game logic. In 
some cases, the lackluster performance 


id Abandons Java for QUAKE 3: ARENA 


ohn Carmack considered using 
Java in id’s games for quite some 
time, ever since he announced that 
the company was leaning towards 
client-downloadable code for the 
Trinity project. “The QA game architecture 
So far has two separate binary .DLLs: one 
for the server-side game logic, and one for 
the client side presentation logic.” Games 
that licensed the Quake 2 engine, notably 
HALF-LIFE and HERETIC 2, also wound up 
using client side .DLLs. However, with the 
hacking attacks on QUAKE 2 servers in 
mind, Carmack states that, “While it was 
easiest to begin development like that, 
there are two crucial problems with Ship- 
ping the game that way: security and 
portability. If we were willing to wed our- 
selves completely to the Windows plat- 
form, we might have pushed ahead, but | 
want QUAKE 3: ARENA running on every plat- 
form that has hardware-accelerated 
OpenGL and an Internet connection.” 

His solution: “I had been working 
under the assumption that Java was the 
right way to go, but recently | reached a 
better conclusion. The programming lan- 
guage is interpreted ANSI C. The game 
will have an interpreter for a virtual RISC- 
like CPU.” UNREAL followed a similar 
approach: companies that license the 





engine can opt to use compiled C or C++ 
code, and interpreted UnrealScript is 
available for homebrew scripting. 

The advantages of using a C or C++ 
subset for your VM are obvious when it 
comes to handling legacy code. 
lronically, it was Java portability prob- 
lems that led id to develop the QuaKE 3 
custom VM. Sun’s promise of “write once, 
run anywhere” did not hold for the 
Invocation API on important server plat- 
forms, so Carmack decided to abandon 
the embedded JVM he had planned to 
use. “My ideal situation,” he stated, 
“would be to include the interpreter in 
the QUAKE3.EXE, and just treat class files 
as data to be loaded and dealt with like 
anything else.” Unfortunately, while this 
solution works fine on Win32 platforms, 
this was not guaranteed for Linux, OS/2, 
or even Solaris. “Having made the deci- 
sion to do my own interpreter, | feel much 
more at ease not having to rely on anyone 
else’s external code. When it comes 
around to the next development cycle, | 
will make the Java decision again.” As for 
embedding: “We are still working with 
significant chunks of an existing code 
base. If | did want to go off and start 
fresh, | would likely try doing almost 
everything in Java.” 
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of Java core classes might force you to 
replace them with your own custom Java 
code, or even use some native code 
instead. In the end, you will have a few 
cleanly separated native code modules 
you can optimize to your heart’s con- 
tent, controlled by robust Java code. The 
object-oriented design propagates top- 
down into your native code, which 
should be another benefit. 


The Holy Grail: 100 Percent Pure 
Java 


S upporting the multitude of 
Internet server platforms (Solaris, 


Linux and other UNIX flavors, OS/2, 
and Windows NT) has become 
increasingly important for multiplayer 
games. Presuming the existence of 
decent Java networking core classes 
and acceptable performance of Java- 
based scene lookup and collision 
detection code, a dedicated server 
implemented entirely in Java is an 
attractive possibility — portable by 
default and, in the absence of JNI, eas- 
ily compiled to native code. 

Portability issues are not as pressing 
for clients, the majority of which are 
Win32-based. At the same time, a real 
need for native code might only be 
found for the client, which is pushing a 
lot of raw data (textures and sounds) 
from local disk to local memory to 
native driver code. Unfortunately, only 
a few games, such as id’s experimental 
QUAKEWORLD release, have separated 
the client and server completely. 
Consequently, a dedicated Java server 
means a separate code base that partly 
duplicates the shared client/server 
sources. Dedicated servers have become 
quite common recently, but in the 
long run, the code duplication is not 
acceptable. Automated Java-to-C or C- 
to-Java conversion might offer a tem- 
porary workaround only. 

Ultimately, shipping a client written 
in Java will require decent Java bind- 
ings around reliable, cross-platform 
APIs, and these are nowhere to be 
found. Java does not have official 
OpenGL bindings, there is not even a 
portable native API for 3D sound, and 
the politics surrounding Sun's propri- 
etary Java3D scene graph API doesn’t 
help matters. For the foreseeable 
future, commercial games will not be 
feasible without JNI and native code. 
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(2Java Today: What's Next? 


ew games (Red Storm Entertain- 

ment’s POLITIKA is among them) 
have shipped with Java built into 
them, but if you want to see a full-sized 
example of an embedded Java VM run- 
ning QUAKE 2 deathmatch right now, 
you should visit the Q2JAVA web site at 
http://www.planetquake.com q2java. 
Q2JAVA, orginally by Barry Pederson, is 
a cooperative open source implementa- 
tion of the QUAKE 2 multiplayer game 
logic and works with the native QUAKE 
2 executables on Windows 95/98/NT, 
Linux and (as a dedicated server) 
Solaris. 

This article has introduced the two 
major roads to using Java for your game, 
though admittedly, a lot of details have 
been omitted and major issues (like 
security) were not touched upon. 


However, Gamasutra.com is hosting 
more of my Java game development 
articles, including a web version of this 
article that includes an annotated list of 
references. & 
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e See the JNI specification at: 

http:/ /java.sun.com/products/jdk/1.2/ 
docs/guide/jni/index.html 

http:/ /java.sun.com/products/jdk/1.1/ 
download-pdf-ps.html 

e See Rob Huebner’s 1999 GDC slides at: 
http: / /www.nihilistic.com/GDC99/Java- 
e Gordon, Rob, et al. Essential JNI: Java 
Native Interface. Prentice Hall Computer 
Books, 1998. 

e See the extended version of this arti- 
cle on Gamasutra.com, which includes 
further links and resources. 














——— 


Fmbedded Java in VAMPIRE: THE MASQUERADE 


narecent developer update, 
Nihilistic’s Director of Tech- 
nology, Robert Huebner, stated: 
“lve always been rather anti- 
Java; all the Internet hype surrounding 
the language was overwhelming. But 
after examining the language further, it 
was clear that it makes an ideal scripting 
language for games. The embedded Java 
API allows us to provide our designers 
with a subset of the Java environment, 
and the JNI interface allows us to pro- 
vide hooks from the Java Virtual Machine 
(JVM) directly into the game engine.” 
The new Java-based system will replace 
a custom compiled language, COG, that 
the team used in its previous title, JEDI 





KNIGHT: DARK FoRCES 2. According to 
Huebner, “The JVM is a lot faster than 
the systems we wrote ourselves; their 
kernel is more heavily optimized, the 
available Java compilers produce much 
more optimized object code, and the 
newest JVM systems include just-in-time 
(JIT) compilation to native instructions 
as a standard feature. And since the lan- 
guage is so much richer than our previ- 
ous C-subset, it gives the designers a 
much wider range of possibilities.” 
Because Nihilistic is developing primari- 
ly for Windows, it is able to apply solu- 
tions that were not feasible for id 
Software’s multi-platform QUAKE 3: 
ARENA Strategy. 
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Rounded landscapes like this can be rendered quickly using 
Bézier patches. 
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n my article last month, I covered the 
basics of curved surfaces, including 


Hermite curves, Bézier curves, and Bézier 


surfaces. I warned that the implementa- 


tions would be straightforward and naive, 


and therefore slow. 


That was last month. The purpose of this article is to take 
what we learned last month and delve into optimization 
strategies and algorithms so that our slow, straightforward 
code can become fast (but still straightforward) code. 

This article ends with a more interesting demo than last 
month’s — a terrain system of Bézier patches. Therefore, this 
article presents information that bridges last month’s infor- 
mation and this month’s terrain system. We'll find a better 
patch tessellator, quickly talk about terrain light map genera- 
tion and why it’s a good idea, and finally talk about putting 
patches together to form the terrain and the bevy of compli- 
cations that accompany it. 


Central Difference Tessellation 


t the end of the last article, I created a demo of a single 

Bezier patch, tessellated uniformly, bright red, that 
spun around at a woeful frame rate. Certainly, this was 
nothing you could use to build a robust application. The 
computations involved for just that one patch included 32 
cubic function evaluations, 32 quadratic function evalua- 
tions, two vector normalizations, a vector cross product, and 
OpenGL lighting for each point. We can do two things to 
speed that up: do less work per point, and, even more 
importantly, calculate fewer points. 

Perhaps the worst sin of my Naive implementations was 
the patch tessellator, UniformPatchTessellator. It just calculated 
a ten-by-ten grid of points on the patch. It calculated each 
point explicitly and just about as Slowly as it could. If we can 
devise a better patch tessellator, we’re a long way towards a 
truly useful implementation. 

There are better tessellation algorithms out there, and I'll 
discuss one particular algorithm that I’ve used to make these 
Bezier patches really crank: central differencing. One form of 
the central differencing algorithm is mentioned in Watt and 
Watt’s Advanced Animation and Rendering Techniques 
When Brian’s not coding up a storm or sleeping through meet- 
_ ings at CogniToy, he’s filling up otherwise useful pages of the 
magazine with his nonsensical ramblings. Keep him busy by 
| writing to him at brian_sharp@cognitoy.com with questions or | 
comments or else he 1 
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(Addison-Wesley, 1992), but this version is modified signifi- 
cantly from the one they present. 

The fundamental mathematical construct used in central 
differencing is the Taylor polynomial. A Taylor polynomial 
of a function is a polynomial that approximates that func- 
tion near a certain point. So, let’s say you’ve got a sine curve, 
and you start atop a peak of the curve. Then, you can form 
polynomials that look like a sine curve near that peak. As 
the polynomial becomes more and more complex, it looks 
more and more like that sine curve. The formula for the 
Taylor polynomial of a function h(x) near a parameter u is: 


h(u + du) = ys 


i=0 

Then, in practice, you pick some limit other than infinity 
for the summation, and you get an approximation of the 
function — the higher the limit, the better the approxima- 
tion. However, in our case, we’re dealing with cubic func- 
tions. Therefore, any derivative after the third will be zero, 
so it’s pointless to make the upper bound anything higher 
than three. So, if our cubic function is h(x), its Taylor poly- 
nomial will look like this: 


h'(u) where Hv is the i” derivative of h. 


2 B| 
h(u + du) = h(u) + du(h’(u)) + ue +h’(u) + me h’”’(u) Eq. 1 


You’ve probably noticed by now that all this does is take 
one cubic polynomial and give us another cubic polynomial. 
You're probably wondering what that does for us. This is 
where central differencing takes over. The idea behind cen- 
tral differencing is that, given information about the end- 
points of a curve segment, we'd like to be able to find the 


LISTING 1. This code takes two points on a cubic curve and 


the second derivatives of the curve at those points and finds 
the midpoint and its second derivative. 





CentralDifference(h(a), h’’ (a), h(b), h’“(b)). 
{ 

// Get the midpoint parameter value. 

mid = (atb)/2; 


// Get the midpoint of the line between 
// h(a) and h(b) 
avgPoint = (h(a) + h(b)) / 2; 


// Find the second derivative of 
// h((atb)/2). Since the second 
// derivative is linear, we can 
// just average them. 

h’’ (mid) = (h’’(a) + h’“(b)) / 2; 


// Find du, the distance from h(a) 
// to h((a+b)/2). 

du = (b-a) / 2; 

// Now h(mid) is this next term 
// plus avgPoint. 

secondlerm = -(du*du)*h’ * (mid) /4; 
h(mid) = avgPoint + secondTerm; 


return h(mid) and h’~ (mid) 
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FIGURE 1. The parameters used in the Taylor polynomials 
for central differencing. 


f (u-du) 
f (u) 





midpoint. That way, we can then recurse on both sides of 
the midpoint and find their midpoints, and so on, until we 
have our curve. 


Rather than keep you hanging, I’ll mention the process 
right now and go through the derivation of it afterwards. If 
we have two points along the function h(x), say h(a) and 
h(b), and we want to find the point halfway between h(a) 
and h(b), or h((a+b)/2), we do the following. First, we aver- 
age h(a) and h(b), which gives us just a point on the line 
between them. 

Next, we take h’’(a) and h’’(b), the second derivative of H 
at a and b. We average them to find the second derivative 
at the midpoint, h’’((a+b)/2). We also find du, which is half 
the parameter distance between h(a) and h(b), or (b-a)/2. 
Then, we compute _du2 *h’’((a+b)/2) / 4. We add that to the 
averaged point we found above, and that’s our point. The 
pseudo-code for this process is shown in Listing 1. 


Central Differencing: The “Why” 


Ss o we’ve established that central differencing is a better 
way to tessellate a patch. Now we need to choose some 
values so that the Taylor polynomial makes this work. We'll 
call the midpoint h(u), and du is the distance from the mid- 
point to an endpoint. That makes the first endpoint h(u-du), 
and the other endpoint h(u+du); Figure 1 shows an example 
curve. From that, we can evaluate the Taylor polynomial as 
though we were solving for the endpoints, given the mid- 
point. That gives us the following two equations: 





GS ee. b> (am 
h(u + du) = h(u) + du(h’(u)) + Sn") ~ “nw 


Eq. 2 | 





2 _af ee 
h(u — du) = h(u) + —du(h’(u)) + \ * h’(u) + ce nw 
Eg. 3 


In Equation 3 we can pull the negative sign out of the —du 
terms on the right side, giving use these two equations: 


24 ap).. £5 
h(u + du) = h(u) + du(h’(u)) + Sn") 4 anu) 
Eq. 4 
—~(du") 


h(u — du) = h(u) + —du(h"(u)) + “ h’(u) + = rt) 


BO. 
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FIGURE 2. One level of recursion in central difference 
patch tessellation. 
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Equations 4 and 5 by themselves do not give us enough 
information to solve for the midpoint h(u), but we can reduce 
them to a single nice equation just by adding them together. 

h(u + du) + h(u — du) = 2h(u) + du*h’(u) te & 
=f 

The positive du and negative du terms cancel out, which is 
why we have such a nice equation. However, we don’t want 
to solve for the endpoints — we need the midpoint. 
Rearranging the equation we get: 


2h(u) = h(u + du) + h(u— du) - du?h’(u) 
Dividing through by 2 and breaking things up, it becomes: 


h(u+du)+h(u-du)  du?h’(u) 
hu) = AE) daw) Eq. 8 

That first big fraction on the tight side of the equation is 
just the average of the two endpoints. That’s easy enough to 
find because we're assuming we have whatever information 
we need about the endpoints. The second fraction on the 
right side is a little harder, though. It depends on the second 
derivative at the midpoint. Can we get that from the end- 
point? Well, note that the second derivative is just a linear 
equation. That’s easy enough to find from the endpoints; we 
can just average the second derivatives of h at the endpoints, 
and we've got it: 


RCs h’(u+ du) ; h’(u — du) Eq. 9 


Now, throwing that back into equation 8, we’ve got: 
iii) = htu+du)+htu—du) — du’(h’(u+du)+h"(u—du)) 
a ee 


Eq. 10 
That’s a mouthful, but that’s it. What that Says is that we 
can find the midpoint by averaging the endpoints and 
adding on a weighted average of the endpoints’ second 
derivatives. Since we know we can find the midpoint’s sec- 
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Eq. 7 Ww 


ond derivative (we just did it in Equation 9) we’ll have 
enough information when we’re done to recurse and find 
the rest of the curve. 

That’s just for a curve, though, as h is only a function of 
one variable. We want do this for patches, which are func- 
tions of two variables. But before we move on, we should 
take note of an interesting property of the central difference. 


Nonlinearity 


i n Equation 10, we find the midpoint by averaging the 

endpoints and adding on the other term, the weighted 
second-derivative term. What’s interesting is that if that 
second term is 0, then it means that the midpoint lies on 
the line between the endpoints. Therefore, we’ll refer to 
that second term as the nonlinear term of the central differ- 
ence — it determines how far the midpoint is from the 
line between the endpoints. 

As you might suspect, this is a great heuristic that we 
can use to decide how far to tessellate a curve. We just set 
a certain threshold, and when the magnitude of the non- 
linear term is below that threshold, we stop recursing and 
return. Therefore, we’ll have much more detail in very 
curved areas, and not nearly as much detail in the less 
curved areas. This is a perfect way to save detail for the 
parts of the curve that need it. We’ll use the nonlinear 
term later on when we put together heuristics for our 
patch tessellation. 


Central Differencing Revisited 


e’ve derived enough to tessellate a curve with cen- 
tral differences. Now we need to figure out how to 
tessellate a patch with central differences. As it turns out, 
central differences in two dimensions are slightly harder 
than the one-dimensional variety. So, we describe our 
problem again. This time, we want to be able to take infor- 
mation about four corner points on a patch and find the 
same information about the midpoints between the cor- 
ners and also the center point. An illustration of this is 
Figure 2. 

Looking at the illustration, it’s clear that if we can find 
the midpoints of this one level given the corners, we can 
recurse within each of the four smaller squares using the 
same method, and so on, until we have our patch. Now, we 
aren't sure exactly what information we need at the corner 
points, so let’s see what it would take to find the other five 
points. 

The u-midpoints can be generated using the one-dimen- 
sional central differencing. We can ignore v, since it stays 
constant, and use the endpoints and their second deriva- 
tives with respect to u to find the u-midpoints and their 
second derivatives with respect to u. 

The v-midpoints can be generated the same way. We can 
ignore u and use the endpoints and their second deriva- 
tives with respect to v to find the v-midpoints and their 
second derivatives with respect to v. 

To keep this all straight, it’s useful to keep track of what 
information we have for which points: 
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O° f (u,v) o°f (u,v) 
due Slo 


corners > f (u,v), 


O°f (U,V) 
. a C ‘ 
u—mids = f (u,v),—_—>— 
Ou 
. Of (u,v) 
v= mids = f (u, Vv), [ > 
Ov 


center > nothing 


Before we worry about finding the center point, we should 
make sure the uv-midpoints and the v-midpoints are complete. 
That is, if we want to recurse, we need to make sure that we 
end up with the same exact information at the new points as 
we got from the corner points. Otherwise, that information 
won't be available to the four smaller squares when we try to 
recurse. So, the objective is to find the second partial deriva- 
tive of f with respect to v at the u-midpoints, and the second 
partial derivatives in u at the v-midpoints. However, it’s not 
clear quite how we do this, since the one-dimensional central 
differencing doesn’t say anything about how to interpolate a 
variable of v along u or vice-versa. A clue comes from the 
equation of the Bézier patch: 


3 3 
3 3 
>, PB; (u)B; (Vv) 
i=. j=0 
Let’s find the second partial in v along u first. We must find 
the second derivative of the patch with respect to v. From the 
above equation, we’d have: 


d°B°(v) 


3 3 
> fe Oa 
70 dv" 

Now, the basis function with respect to U, B(u), is a cubic. 
Since it’s untouched, we can see that the second derivative of 
the patch with respect to v is a cubic function in u. If we have 
any cubic in u, we can interpolate it in the u direction using 
the one-dimensional central differencing. 

In this case, then, the function we’re interpolating is the 
second derivative with respect to v: 


O° f (u,V) 


7(U) = ; 
. ov" 


So, we can interpolate g(u) if we have g(u) at the endpoints 
and ¢’s second derivative with respect to u at the endpoints. 
That means that to get the second partial derivative in v at the 
u-midpoints, the corner points need this data: 


O°f(u,v) of (u,v) 
ow | aura’ 


The corners already have the first value, the second deriva- 
tive with respect to v. We have to add the second one, the sec- 
ond partial derivative with respect to u of the second partial 
with respect to v. 

Now we've got the second partials in v at the u-midpoints. 
We can easily find the second partials in u at the v midpoints 
in the same way if we just swap the variables. So, the second 
partial in w is a cubic function in v, so we can find it by adding 
the following to the corner points: 


o°f(u,v) O°f (u,v) 
ou? ov? ou" 


Again, the corners already have the first item. Also, we can 
switch the partials around, so: 
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O*f(u,v) _ of (u,v) 
ov-au= — au’ av” Eq. 11 
Finally, we note that this term is a linear function of u and 
v, sO we can get it at the u-midpoints and the v-midpoints just 
by averaging the values from the corner points. 
Therefore, we have all the information we need in the cor- 
ners, and we can get that information back at the u-midpoints 
and v-midpoints. Just to keep track, here’s what we’re up to: 


a’ f(u,v) a’f(u,v) of(u,v) 


corners > f (U,V);—= 7 ——, — 
Ou" Ov Ou’ ov" 
2¢ 2 4¢ 
u—mids > fo ei ee 
oue ov" du dv~ 
2 OW 4¢ 
v—mids > tinh 2 Le SEY) 
oun ov Ou" ov~ 


center — nothing 


Now, we just have to fill in the center point, and we're 
done. (Don’t worry, we’re almost there.) 

Referring back to Figure 2, the center point can be derived 
from the u-midpoints just as each of the v-midpoints was 
derived from its two corner points. So we use the u-midpoints 
and their second partials in v to find the center point and its 
second partial in v. Then, we can get the center’s second par- 
tial in u by averaging the v-midpoints’ second partials in u. 
Finally, we can get the center point’s final value, the second 
partial in v of the second partial in u (the value from Equation 
11) by averaging the values from the v-midpoints. 

We're done! It took a while, but we finally have all the 
information for all the points: 


26 ) 2f ) 4¢ ; 
corners > fis 9 a f(uv) dfuv) 


a Ov” uv" 
2£7 : Sp, 4 
u—mids > Pi d Pye) oY) 
due ov" Ou’ ov" 
OI es ;  e ory ; 
v—mids > Hap te TD ee 
oun ov" ou’ Ov" 
2 ) 2£ ) 4 ; 
center > f(u, og a) eae eT 
due av?" awav 


Therefore, given four corner points with those five values, 
we can create the u-midpoints, v-midpoints, and center point 
with the central differencing. From there, we can recurse into 
the four smaller squares formed by the new points to contin- 
ue tessellation. Pseudo-code for doing this is shown in Listing 
2. We start by explicitly computing this information for the 
four patch corner points, and then recurse. That’s great, but 
there’s still something missing. We’ve proved the recursive 
step, but we don’t have a base Case. When do we know we're 
done? How far do we recurse? 


Recursion Heuristics 


Way already mentioned a very powerful tool for deciding 


when to stop recursing. The nonlinearity term from 
the central differences is an estimation of how curved the sur- 
face is at the spot we're tessellating. We can use the nonlinear- 
ity terms from the generation of the midpoints to determine, 
then, the curvature of the subpatch we’re working with. Then, 
[way 
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FIGURE 3. The nonlinearity vector 
lies roughly in a slice of the frustum. 


nonlinearity 
ete 


given some threshold, if the magnitude 
of the nonlinearity terms for all the 
midpoints was less than the threshold, 


| we'd just leave the subpatch as the two 


triangles formed by the corners and 
return. So if the threshold is very high, 
we end up with a very low-detail patch 
of just two triangles. If the threshold is 
too low, the patch is extremely detailed, 
which will look good but run slowly. 
Hence, picking a good threshold value is 
just a matter of testing, and would be a 
good candidate for an options screen for 
the game player. 

However, just because a surface is 
very Curvy doesn’t mean we want it to 
continue tessellating. If the surface is 
SO miles away from the camera and 
occupies a three-by-three block of pix- 
els on the screen, we’d rather it be very 
low detail. However, if we zoomed in 
with a sniper rifle (think GOLDENEYE) to 
look at that terrain, it might be 50 
miles away, but might now occupy a 
200-by-200 block of pixels, in which 
case we'd like it to tessellate quite a bit. 
So, we need a way to add distance and 
field-of-view angle into our heuristic, 
and we’d like it to be fast. Ideally, we’d 
like to know how big the nonlinearity 
vector is in pixels. 

Here’s a good, albeit rough, heuristic. 
Consider the camera frustum. The non- 
linearity vector lies roughly in some 
slice of the frustum perpendicular to 
the screen (see Figure 3). Let’s say we 
can figure out how wide, in pixels, that 
Slice of frustum is. Then the ratio of the 
magnitude of the nonlinearity vector 
to the width of the frustum slice is the 
(very rough) size of the nonlinearity in 
screen space, expressed as a percentage 
of the screen width. For instance, if the 
patch was so curvy that the nonlineari- 
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FIGURE 4. The trigonometric deriva- 
tion of the width of the frustum slice. 


width of frustum slice 


nonlinearity - 
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ty vector stretched all the way across 
that slice of frustum, the ratio would 
be 1. More likely, the nonlinearity vec- 
tor will be something relatively small, 
and the ratio will be something like 
0.002, which is 0.2 percent of the 
screen, equal to about five pixels in a 
640x480 screen display. 

However, to do this, we still need to 


LISTING 2. 


// Find the midpoints and center point given these four cor 


find the width of that frustum slice. 
So first we take the vector from the 
eye to a point near the nonlinearity 
vector, like one of the corner points. 
Call that vector d, for distance. Next 
we need the field-of-view angle, and 
then we’re all set to do a little 
trigonometry. Referring to Figure 4, 
we know one angle of the triangle, 
and d is one of the legs, so the other 
leg is half the width of the frustum 
Slice. So that leg’s length is d * 
tan(fov/2). Double that, and we have 
the width of the frustum slice. Then 
take the ratio of the nonlinearity vec- 
tor’s magnitude to the frustum slice 
width, and we have our heuristic. 
Keep in mind that all of this is very 
rough. Since it’s all about the visual 
quality, all of these heuristics are very 
open to tweaking. For instance, in my 
terrain demo, I decided that I cared 
more about detail of terrain close to 
the camera than far away, so I squared 
the distance attenuation part so it fell 
off faster and I could devote more tri- 


This code takes four patch corners and a number of derivatives at those 
corners and finds the midpoints and center points. 















// values, u0vO is the point (the zeroth derivatives w/ respect to 


// is the second partial in u, u0v2 is the second partial in \ an 
// second partial in u of the second partial inv 


DifferencePatch(cO, ct, c2, c3) 
{ 


// Here are the points were trying to find. 


points umid0, umidi, vmid0, vmidt, cen; 


// Find the u-midpoints’ u0vO and u2v0 through Central Differencing. aa 
umid0.u0v0 and umid0.u2v0 = CentralDifference(c0.u0v0, c0.u2v0, c1.u0v0, c: 
umidi.uOv0 and umidi.u2v0 = CentralDifference(c2.udv0, c2.u2v0, c3.u0v0, c’ 







/1 Find the v-nidpoints’ u0v0 and udv2 through Central Differencing. = 
vmid0.u0v0 and vmid0.u0v2 = CentralDifference(c0.udv0, c0.u0v2, cl.u0v0, ct.udv2, 
vmidi.u0vO and vmidi.u0v2 = CentralDifference(c2.u0v0, c2.u0v2, c3.u0v0, 3.U 


// Find the u-midpoints’ u0v2 and u2v2 through Central Differetcing. = = 
umid0.u0v2 and umid0.u2v2 = CentralDifference(cO.u0v2, c0.u2v2, cl.u0v2, cL.u2v2); 
umidi.u0v2 and umid1.u2v2 = CentralDifference(c2.u0v2, c2.u2v2, c3.u0v2, c3.u2v2); 


// Find the v-midpoints’ u2v0 and u2v2 through Central Differencing. 





vmid0.u2v0 and vmid0.u2v2 = CentralDifference(c0.u2v0, c0.u2v2, c1.u2v0, cl.u2v2); oe 
vmidi.u2v0 and vmidi.u2v2 = CentralDifference(c2.u2v0, c2.u2v2, C3-U2v0, C3022); 


// Now we just have to find the center point. 


Find it from the midpoints. 


cen.u0v0 and cen.u2v0 = CentralDifference(vmid0.u0v0, vmidO.u2v0, vmidi.udv0, 


vmid1 .u2v0) ; 


cen.u0v2 and cen.u2v2 = CentralDifference(umid0. u0v2, umid0.u2v2, umidt . u0v2, 


umidi .u2v2) : 


// We’re done! 


hx 
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angles to the patches up front. In addi- 
tion, while it’s possible to sit down and 
hash out the ideal threshold value for 
different performance levels, you'll 
probably have more luck (and spend 
less time) just making the threshold 
modifiable at run time and tweaking it 
to suit your needs. 


Filling in the Cracks 


ne of the dark secrets of this 

dynamic tessellation that we 
haven’t mentioned is that it causes 
cracking in the patch. When two sub- 
patches that share an edge are tessellat- 
ed differently, there will be a hole 
between them, as shown in Figure S. In 
practice, the cracking is very evident 
and distracting, so it can’t just be 
passed off as an acceptable visual glitch. 

At first glance, one might be tempted 
to go with a naive fix, like just filling the 
cracks with triangles. The problem is, if 
the higher-detail surface is, say, two lev- 
els of detail higher, the hole won't be 
triangular, but pentagonal. It turns out 
that it’s easier and faster to fix the cracks 
in a better way than that. 

Cracking is caused when subpatches 
that share an edge are of different lev- 
els of detail. Therefore, the fix for the 
problem is to make it so that shared 
edges are always at the same level of 
detail. In my terrain demo, I use a post- 
pass over the vertices to accomplish 
this. For each patch, I check to make 
sure that the edges of its four subpatch- 
es are of the same level of detail. If one 


FIGURE 5. An edge shared by sub- 
patches at different detail levels 
exhibits an open triangular hole. The 
z value of the extra point in the red 
patch doesn’t line up with the blue 
patch’s edge. 
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of them isn’t at the same level of detail, 
I fix it by collapsing the middle vertex 
of the higher-detailed edge into the 
corner, as in Figure 6. Then | recurse on 
the four subpatches. While this does 
require another pass over the data, it 
works surprisingly well. 

Now that we are tessellating patches 
dynamically with no cracking, we're 
getting closer. But we're not there yet. 
As far as we know, the patch is still a 
flat, unlit color. Next, we’d like to make 
it look a bit better. 


Textures and Light Maps 


© ne of the best ways to improve 
the visual quality of a patch is 
through texturing and lighting. The 
first thing we need are texture coordi- 
nates so we know how to place the tex- 
ture onto the patch. The simplest 
approach, sufficient for many purposes, 
is just to use the patch (u,v) as the tex- 
ture coordinates. We can get these easi- 
ly from the central difference tessella- 
tion by averaging the (u,v) of the 
corners to get those of the midpoints 
and center point. 

Given those, it’s easy to drop a tex- 
ture onto the patch. We just toss a ter- 
rain texture of choice at OpenGL, and 
then use the patch (u,v) at each point as 
texture coordinates. This maps the tex- 
ture directly across the surface. But a 


texture with no lighting still doesn’t 
look very good. 

It might be tempting to use OpenGL's 
lighting as we did with the 
UniformPatchTessellator. However, there are 
a number of reasons not to do this. kor 
one, finding the normal to the surface 
at each point requires a fair amount of 
work, including the two vector normal- 
izations and cross product. Furthermore, 
the central differencing is already com- 
plicated enough without having to 
interpolate the normals. And there's 
one more reason not to use OpenGL 
lighting: OpenGL’s per-vertex lighting 
makes the dynamic tessellation of our 
terrain very obvious. When the detail 
level of the terrain changes, the lighting 
shifts disturbingly, and when the poly- 
gon count in the terrain is low, the 
lighting looks stretched and linear from 
the Gouraud shading. 

The solution is to use light maps. (For 
a more thorough discussion of light 
maps, see “Multitexturing in DirectX 6 
by Jason Mitchell et al. [September 
1998]). Light maps are handy not only 
because they not change with the tes- 
sellation, but they also actually give the 
illusion of more detail. Even if the sur- 
face is made of two triangles, the light 
map still depicts smooth, curved light- 
ing playing across the curves that aren't 
really there. 

In my terrain demo, | didn’t particu- 
larly want to pre-calculate my light 





FIGURE 6. By collapsing the center vertex into the corner, the cracking is fixed. 







FIGURE 7. Terrain Texture * Terrain Light Map = Lit Terrain. 
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LISTING 3. This code uses UniformPatchTessellator and OpenGL feedback mode to 
make a light map for the terrain. 





// it orthographic to save on transformation cost. 
::glMatrixMode( GL_PROJECTION ); 
::glLoadIdentity (); — 
::gl0rtho( -10000, 10000, -10000, 10000, 1, 20000); 


:iglMatrixMode( GL_MODELVIEW ); 
::glLoadIdentity (); : 
-gluLookAt( 0,0,10000, 0,0,0, 1,0,0 ); 


::glClear( GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT ); 


// Position the Light. 

::glEnable(GL_LIGHTING) ; 
float position[] = {20.0f,0.0f,5.0f, 0.0f}; 
::gllightfv(GL_LIGHTO, GL_POSITION, position): 


// Tessellate our patch. 
generatePoints( size, controls, basesU, basesV ): 


// Kick OpenGL into feedback mode and render the patch. 
float* feedBuffer = new float[ size * size * 8 1; 


::glFeedbackBuffer( size * size * 8, GL_3D_COLOR, feedBuffer ); 
: iglRenderMode( GL_FEEDBACK ): 


‘ pColorsi( 1, 1 1). 
: :glBegin( GL_POINTS ); 
for ( int v = 0; v < size; y++ ) 
{ 
for’( int u= 0: u-< size: get ) 
{ 
::glNormal3fv( norms + (u + (vksize))*3 ); 
::glVertex3fv( verts + (u + (v*size))*3 ); 
} 
} 
:iglEnd(); 


// Read the data out of the buffer. 
int count = ::glRenderMode( GL_RENDER ye 


int texPos = 0; 
float* texData = new float[ 3 * size * size te 


for(int x = 0; x. < count’ x44) 


{ 
if ( feedBuffer[ x ] == GL_POINT_TOKEN ) 
{ 
texData[ texPos + 0 ] = feedBuffer[ x +4 1 
texData[ texPos + 1 ] = feedBuffer[ x +5 ie 
texDatal texPos + 2 ] = feedBuffer[ x + 6 1. 
texPos += 3: 
oe 
} 
else 
{ 
Std::cout << "ERROR parsing feedback buffer array." << std::endl; 
delete[] texData; 
return 0; 
} 
} 


Continued on p. 47. 
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// We want a huge projection so ‘that it won't clip anything, and we want _ — 





maps, and I didn’t want to implement 
my own lighting model, either. 
Therefore, I found a handy solution that 
generates fairly good-looking results: 
OpenGL feedback mode. 

The idea here is that at patch cre- 
ation time, we want to sample the 
lighting at a number of points evenly 
spaced across the surface, and use their 
lit colors as texels in a light map. It 
turns out that even a fairly small light 
map will look just fine. Since we want 
the points evenly spaced, we’ll use 
UniformPatchTessellator again to generate 
an 8x8 grid of points the slow way, 
light them, and build a texture out of 
it. The code for this is shown in Listing 
3. The process is surprisingly fast — a 
single patch’s light map takes around 2 
or 3 milliseconds to generate and 
upload to OpenGL. 

Figure 7 shows the end product. We 
have the textured pass, the lit pass, and 
when combined, the textured and lit 
patch. Quite an improvement, indeed. 
If I'd promised a terrain system, though, 
a single patch is hardly sufficient. We 
need to take this good-looking patch 
and somehow come up with a way to 
define an entire landscape. 


Multiple Patches 


ie aking terrain out of multiple 
patches doesn’t sound like any- 


thing monumental. After all, we could 
just create a five-by-five grid of patches 
and draw them. Unfortunately, albeit 
not surprisingly, it’s not that easy. 

The first and most obvious problem is 
that if we just generate a 4x4 grid of ran- 
dom control points for each patch, the 


FIGURE 8. For the curves to connect 
smoothly, their endpoint tangents 
must both point in the same direction. 
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LISTING 3. (continued from page 46.) 


// Make it into a texture. 
unsigned int texNum; 
glGenTextures( 1, &texNum ); 


qiBindTexture( GLTEXTURE.20, texNum); 


// Set the tiling mode. 


// Set the filtering. 


glTexParameteri(GL_TEXTURE_20, GL_TEXTURE_MAG_FILTER, GL_LINEAR) ; : 


We don “t want lightmaps to repeat, or we 
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP); — 
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP); 



































glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR_M NA _INEAR) ; 


gluBuild2DMipmaps( GL_TEXTURE_20, 3, size, size, GL.RGB, GL_FLOAT, texData ); 


delete[] texData; 


return texNum; 


patches will line up in x and y, but the z 
values of patches along shared edges 
will have nothing to do with each 
other. The terrain will have gaping 
holes where patches touch. 

We could make sure that the patches 
share the same edge points so that there 
wouldn’t be any surface breaks between 
them. However, that’s still insufficient — 
the Bézier patch representation doesn't 
guarantee anything about continuity 
between patches. We have to manually 
make sure that we preserve a number of 
conditions or else the seams between 
patches will likely be sharply discontin- 
uous, not at all like real terrain. 


Continuity Conditions 


W e need to ensure that the tan- 
gent vectors of patches along 
shared edges are the same, so that the 
terrain will look smooth, even between 
patches. A property of Bézier patches is 
that the tangent vectors at the edge 
points are defined by the edge control 
points and the control points one level 
in. So the tangent vector in the u direc- 
tion at the upper-right corner is the vec- 
tor through the corner point from the 
point one to its left. 

To preserve continuity between 
patches, we need to ensure that the for 
every control point along a shared edge, 
the tangent vectors are the same. The 
three points involved in those two tan- 
gents consist of the control point itself, 
plus the control points to either side of 
it. If we make sure that those three 
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points lie on a line, we know that our 
terrain will be smooth. 

Figure 8 shows an example of this 
concept. The tangent vectors of the 
joining curves pass through the two 
nearest control points of each curve. If 
we don’t want a crease between the 
curve, both tangent vectors have to 
point in the same direction. 

The terrain demo generates evenly 
spaced x and y values for a grid of con- 
trol points. Then I generate z values 
using a fractal terrain algorithm. After 
that, I do a post-pass on the points to 
move some of them to make sure that 
this condition holds. 

It’s true that this continuity scheme is 
pretty limiting. For instance, if you have 
a patch surrounded by other patches, its 
entire border and all the points one in 
are completely defined by the other 
patches. That’s every point in the patch! 
If we think of the terrain as a chess- 
board, where each square is a patch, 
then after we define all the black-square 
patches, the white-square patches are 
already completely defined. This does 
mean that it’s harder to have very local 
control over the terrain, but it’s still 
quite possible to make nice-looking ter- 
rain. There are different, more relaxed 
continuity conditions, but they're more 
complex than what I’ve discussed. A 
description of these conditions can be 
found in Faux and Pratt’s book in the 
references at the end of this article. 

With this implemented, we can gen- 
erate an array of control points and 
make patches from it, and know that it 
won’t have any sharp seams or visible 


edges. Hypothetically, this could be a 
complete terrain system: a large array of 
textured, lit patches. What we haven't 
yet discussed though, are the speed con- 
siderations. If we leave it like this, we’ll 
be drawing every patch, even those off- 
screen, and while the tessellation and 
drawing are fast for a single patch, if 
we’re drawing a couple hundred patch- 
es, the frame rate will be abysmal. We 
need some way of drawing only those 
patches that are visible. 


Camera Frustum Culling 


ast month, I mentioned that the 
convex hull of a Bézier patch’s con- 
trol points is a good bounding volume 
for the patch. Therefore, if we make a 
bounding box out of the control points 
by taking the minimum and maximum 
x, y, and z values, we can use the 
bounding box to cull the patch. We still 
need a way to tell whether the box is 
inside the camera frustum, though. 

For my terrain demo, I use an object, 
ClipVolume, that takes information about 
the camera and builds six Plane objects 
out of it: the left, right, bottom, top, 
near, and far planes. Planes are capable 
of telling you whether a point is inside 
or outside of them. Therefore, if all the 
points of a bounding box are outside 
any one of the planes, the box is com- 
pletely outside the frustum. So, at the 
beginning of the frame, I build a 
ClipVolume out of the camera by running 
through the patches and testing their 
bounding boxes against the ClipVolume. 
Those that pass are drawn. 

It’s simple, and definitely an 
improvement, but it’s still not fast 
enough. The problem is that we have 
to touch every patch in the terrain 
each frame, so if we have an NXN grid 
of patches, the running time of our 
culling is O(N7). That’s not ideal, 
because we don’t want the number of 
patches to heavily affect the frame rate. 
We need a way to find the visible 
patches without testing every single 
patch’s bounding box. 





L 


Quadtree Terrain Storage 


A s luck would have it, there are a 
number of ways to do this. I used a 
quadtree data structure for the terrain 
demo. A quadtree is a tree where each 
1999 
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node has four branches. In this case, the 
patches are the leaf nodes. Then, the 
nodes at the next level of the tree each 
contain a 2x2 group of patches. They’re 
contained in 2x2 groups by the nodes at 
the next higher level, and so on. 
Eventually, we have our top node, 
which holds the whole tree. 

Each node contains a bounding box, 
which is the box that contains all of its 
sub-nodes. This makes it easier for us to 
cull the terrain. We start at the top 
node, and if it’s within the frustum, we 
recurse on each of its four branches. If 
any node is ever outside the frustum, we 
just return and don’t bother considering 
any of its sub-nodes, as they’re all out- 
side the frustum, also. This way, we can 
reduce our running time for culling an 
NXN grid of patches to O(lgN), which is a 
pretty tremendous win over O(N7). 

To build one of these structures, we 
Start with the patches, the leaf nodes. 
They have their bounding boxes. Then, 
out of each 2x2 block of patches, we 
build a higher-level node. The node’s 
bounding box is the box that contains 
each of the patches’ bounding boxes; 
we just take the minimum and maxi- 
mum x, y, and z from each of the four 
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of those second-level nodes, we use the 
same algorithm to generate third-level 
nodes from 2x2 blocks of second-level 
nodes. We continue until we only have 
one node, and that’s our top node. 

Now we've got a smooth, textured, 
lit terrain culling quickly against the 
camera and drawing correctly. We’re 
almost done. There’s one last glitch 
that we have to take care of. 


Crack Fixing Revisited 


WW: fixed the cracking that occurs 
within a patch, but unfortu- 
nately, cracking also occurs along the 
edge between two patches when one 
patch is tessellated to a different level 
from the other. Fortunately, inter- 
patch cracking is the same kind of phe- 
nomenon as the internal patch crack- 
ing, and we can use the same approach 
to fix it. When we’ve finished tessellat- 
ing the patches we can see, we just 
have each patch check its right and 
bottom edges against the patches that 
share those edges. 

There is a catch, though. We have to 
make sure that we fix the inter-patch 
cracking before we do the internal 
patch crack fixing. The reason for doing 
so is a little confusing at first. We know 
that when we do the inter-patch crack 
fixing, we can collapse edge vertices 
into other edge vertices. We also know 
that when we fix the internal patch 
cracking, we can collapse non-edge ver- 
tices into either other non-edge vertices 
or potential edge vertices. We know 
that internal crack fixing will never 
move an edge vertex, but it may col- 
lapse a vertex into an edge vertex. On 
the other hand, inter-patch crack fixing 
does move edge vertices. 

What does this mean? Imagine an 
arbitrary vertex on the edge of a patch. 
We'll call it V. If we fix the internal 
cracks first, we might collapse a vertex 
into V. Now when we fix the inter- 
patch cracks, we might collapse V into 
something else. When we do this, we 
leave the vertex that was collapsed 
into V hanging where V no longer is. 
This warps the patch and stretches a 
Visible hole in the patch. 

If we fix the inter-patch cracks first, 
we'll collapse V into a corner. Then, 
when we fix the internal cracks and we 
collapse that vertex into V, it will get 
sent to V’s new, correct location. 


Things Yet to Be Optimized 


Tk explanation is certainly more 
difficult than the material present- 
ed in last month’s article. Nonetheless, 
we're done. The terrain demo uses all of 
the things we’ve discussed, from central 
differencing, texturing, lightmapping, 
and crack fixing to quadtree storage 
and camera culling. What’s more, it 
looks good and runs fast. 

That’s not to say that it’s done. 
There’s still plenty of work left to be 
done with the terrain system. For 
instance, the central differencing 
threshold can cause abrupt “popping” 
when the camera gets close enough to 
merit a higher level of detail. Perhaps 
a system where the change was grad- 
ual, using two thresholds, one where 
the detail began changing and one 
where it was done changing, could 
help. Linearly interpolating the size of 
the nonlinearity vector over that peri- 
od would prevent abrupt changes in 
the terrain. 

Another aspect to work on is code 
optimization. While the algorithms 
themselves are fairly fast, the code is 
written to be legible instead of speedy. 
Hand-tuning some of the functions, 
such as the CentralPatchTessellator’s 
tessellation function, could speed up 
the process. 

Furthermore, the current system is a 
bit of a memory hog. It uses 2D arrays 
for all the terrain, even though much of 
the data is unused, depending on the 
level of the tessellation and how much 
of the terrain is visible at any given 
time. A better memory management 
system could speed the system up by 
increasing cache coherency, and allow 
larger terrain sets within reasonable 
memory limits. 

Of course there are aesthetic 
improvements and expansions that are 
begging to be made — a physics sys- 
tem, some wildlife, or some ponds 
would add quite a bit. The code is 
available from my web site (see the ref- 
erence at left), and I invite all those 
who are interested to download it and 
make any improvements they can 
think of. 

Hopefully, between last month’s arti- 
cle and this one, I’ve given a solid intro- 
duction to one of the Next Big Things 
in 3D game engines. Send questions, 
comments, or cool modifications to my 
code, to me via e-mail. 
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Postmortem: 


Looking Glass's 
THIEF: THE DARK PROJECT 


by Tom Leonard 





HIEF: THE DARK PROJECT is One of those games that 


almost wasn’t. During the long struggle to store 






shelves, the project faced the threat of cancellation twice. A 


fiscal crisis nearly closed the doors at Looking Glass. During 
one seven-month span, the producer, project director, lead 
programmer, lead artist, lead designer and the author of our 
renderer all left. Worse still, we felt a nagging fear that we might make a 
game that simply was not fun. But in the end, we shipped a relatively 
bug-free game that we had fun making, we were proud of, and that we 


hoped others would enjoy. 


The Concept 


he THIEF team wanted to create a first-person 
game that provided a totally different gaming 
experience, yet appealed to the existing first-person 
action market. THIEF was to present a lightly-scripted 
game world with levels of player interaction and 
improvisation exceeding our previous titles. The 
team hoped to entice the player into a deep engage- 











_of the game. He lives in the Boston area. Tom has been at Looking Glass for three-and-a-half years, prior to 
which he spent seven years working on C++ development tools at Zortech and Symantec. He is currently 
_working on next-generation technologies, and can be reached at toml@Iglass.com. 
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Stealth is one of your best weapons in TuiEF. The game’s designers made sure that expert players would have to make effec- 
tive use of silent weapons such as the blackjack and the bow and arrow. 


ment with the world by creating intelligible ways for the 
world to be impacted by the player. 

The central game mechanic of THIEF challenged the tradi- 
tional form of the first-person 3D market. First-person shoot- 
ers are fast-paced adrenaline rushes where the player possess- 
es unusual speed and stamina, and an irresistible desire for 
conflict. The expert THIEF player moves slowly, avoids con- 
flict, is penalized for killing people, and is entirely mortal. It 
is a game style that many observers were concerned might 
not appeal to players, and even those intimately involved 
with the game had doubts at times. 

The project began in the spring of 1996 as “Dark 
Camelot,” a sword-combat action game with role-playing 
and adventure elements, based on an inversion of the 
Arthurian legend. Although development ostensibly had 
been in progress on paper for a year, THIEF realistically began 
early in 1997 after the game was repositioned as an 
action/adventure game of thievery in a grim fantasy setting. 
Up to that point we had only a small portion of the art, 
design, and code that would ultimately make it into the 
shipping game. Full development began in May 1997 with a 
team comprised almost entirely of a different group of peo- 
ple from those who started the project. During the following 
year, the team created a tremendous amount of quality code, 
art, and design. 

But by the beginning of summer in 1998, the game could 
not be called “fun,” the team was exhausted, and the project 
was faced with an increasingly skeptical publisher. The 
Looking Glass game design philosophy includes a notion that 
immersive gameplay emerges from an object-rich world gOv- 
erned by high-quality, self-consistent simulation systems. 
Making a game at Looking Glass requires a lot of faith, as such 
systems take considerable time to develop, do not always 
arrive on time, and require substantial tuning once in place. 
For THIEF, these systems didn’t gel until mid-summer, fifteen 
months after the project began full development, and only 
three months before we were scheduled to ship. 

When the game finally did come together, we began to 
sense that not only did the game not stink, it might actually 
be fun. The release of successful stealth-oriented titles (such as 
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METAL GEAR SOLID and COMMANDOs) and more content-rich 
first-person shooters (like HALF-LIFE) eased the team’s concerns 
about the market’s willingness to accept experimental game 
styles. A new energy revitalized the team. Long hours driven 
by passion and measured confidence marked the closing 
months of the project. In the final weeks of the project the 
Eidos test and production staff joined us at the Looking Glass 
offices for the final push. The gold master was burned in the 
beginning of November, just in time for Christmas. 

In many ways, THIEF was a typical project. It provided the 
joys of working on a large-scale game: challenging problems, 
a talented group of people, room for creative expression, and 
the occasional hilarious bug. It also had some of the usual 
problems: task underestimation, bouts of low morale, a 
stream of demos from hell, an unrealistic schedule derived 
from desire rather than reality, poor documentation, and an 
insufficient up-front specification. 

However, THIEF also differed from a number of projects in 
that it took risks on numerous fronts, risks that our team 
underappreciated. We wanted to push the envelope in 
almost every element of the code and design. The experi- 
mental nature of the game design, and the time it took us to 
fully understand the core nature of that design, placed spe- 
cial demands on the development process. The team was 
larger than any Looking Glass team up until then, and at 
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Looking Glass Studios Inc. 
Cambridge, Mass. 

(617) 441-6333 
nttp://www.lelass.com 

Release date: December 1998 

Intended platform: Windows 95/98 

Project budget: Aproximately $3 million 

Project length: 2.5 years 

Team size: 19 full-time developers. Some contractors. 

Critical development software: Microsoft Visual C++ 5.0, 
Watcom C++ 10.6, Opus Make, PowerAnimator, 3D Studio Max, 
Adobe Photoshop, AntimatorPro, Debabilizer, After Effects, 
and Adaptive Optics motion-capture processing, 
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times there seemed to be too many 
cooks in the kitchen. Reaching a point 
where everyone shared the same vision 
took longer than expected. A philoso- 
phy of creating good, reusable game 
engine components created unusual 
challenges that didn’t always fit well 
with schedule and demo pressures. The 
many risks could have overwhelmed 
the project, if not for the dedication, 
creativity, and sacrifices of the team. 
Throughout the life of the project, 


more than 50 
people worked in 
one way or anoth- 
er on THIEF — 
some as part of 
the “Camelot” 
project, others as 
part of the 
Looking Glass 
audio-visual and 
technology sup- 
port staff, some as 
helpful hands 
from other 
Looking Glass 
projects, ‘The core 
team consisted of 
a number of veterans 
of previous Looking Glass titles 
(UNDERWORLD I and II, SYSTEM SHOCK, 
FLIGHT UNLIMITED, TERRA NOVA, BRITISH 
OPEN CHAMPIONSHIP GOLF, and the 
unpublished STAR TREK: VOYAGER), as 
well as some new industry arrivals. The 
project had a number of very talented 
people and strong wills. Although it 
took some time for the team to unite as 
a tight-knit creative force, the final six 
months were incredibly productive, 
spirited, and punishingly fun. 


What Went Right 


DESIGNING DATA-DRIVEN TOOLS. Our 

@ experience on previous titles 
taught us that one of the impediments 
to timely game development is the 
mutual dependence of artists, designers, 
and programmers at every development 
stage. One of the development goals for 
the Dark Engine, on which THIEF is 
built, was to create a set of tools that 
enabled programmers, artists, and 
designers to work more effectively and 
independently. The focus of this effort 
was to make the game highly data- dri- 
ven and give non-programmers a high 
degree of control over the integration 
of their work. Media and game systems 
were to be easily and intuitively 
plugged in and edited by the team 
members responsible for their creation, 
without requiring the direct involve- 
ment of programmers. 

The Dark Object System stood at the 
heart of our strategy. Primarily 
designed by programmer Marc “Mahk” 
LeBlanc, the Object System was a gen- 
eral database for managing the individ- 
ual objects in a simulation. It provided 
a generic notion of properties that an 
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object might possess, and relations that designers to specify most of the behav- designers in order to tweak the game 
might exist between two objects. It iors of the game without scripting or encouraged programmers to empower 
allowed game-specific systems to create programmer intervention. Addition- the designers thoroughly. 

new kinds of properties and relations ally, the relative ease with which vari- The second major component of our 
easily, and provided automatic support ables could be made available to strategy was our resource management 
for saving, loading, versioning, and 


editing properties and relations. It also 
supported a game-specific hierarchy of 
object types, which could be loaded, 
saved, and edited by designers. 
Programmers specified the available 
properties and relations, and the inter- 
face used for editing, using a set of 


straightforward classes and structures. 
Using GUI tools, the designers specified 
the hierarchy and composition of game 
objects independent of the program- 
ming staff. In THIEF there was no code- 
based game object hierarchy of any 
kind. 
Although the implementation of the 
system was much more work than we 
expected, and management of the 
object hierarchy placed significant 
demands on lead designer Tim 
Stellmach, it turned out to be one of en 
the best things in the project. Once 
the set of available properties and rela- 
tions exposed by programmers was Hand-to-hand combat is sometimes necessary. 
mature, the Object System allowed the 
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at run time. 
SOUND AS A GAME DESIGN FOCUS. 

@ Sound plays a more central role 
in THIEF than in any other game I can 
name. Project director Greg LoPiccolo 
had a vision of THIEF that included a 
rich aural environment where sound 
both enriched the environment and 
was an integral part of gameplay. The 


part of gameplay, both as a way that 
players can reveal themselves inadver- 
tently to the Als and as a tool for players 
to distract or divert an Al. Moreover, Als 
communicated with each other almost 
exclusively through sound. AI speech 
and sounds in the world, such as the 
sound of swords clashing, were assigned 
semantic values. In a confrontation, the 








system. The resource management sys- 
tem gave the game high-level manage- 
ment control of source data, such as 
texture maps, models, and digital 
sounds. It helped manage the game’s 
use of system memory, and provided 
the data flow functions necessary for 
configuration management. 

Looking Glass’s previous resource 
management system provided similar 
functionality, but identified resources 
by an integer ID and required a special 
resource compilation step. This tech- 
nique often required recompilation of 
the game executable in order to inte- 
grate new art, and required that the 
team exit the game when resources 
were published to the network. The 
new system referred to a resource by its 
file name without its extension, used a 
file system directory structure for 
namespace management, didn’t leave 
files open while working, and required 
no extra compilation step. Developers 
simply dropped art into their local 
data tree and started using it. To 
expose art to the rest of the team, lead 
artist Mark Lizotte just copied art into 
the shared project directories. 
Compound resources were treated as 
extensions to the file system and were 
built using the standard .ZIP format. 
This allowed us to use off-the-shelf 
tools for constructing, compressing, 
and viewing resource files. The system 
facilitated content development by 
allowing programmers, artists and 
designers to add new data to an exist- 
ing game quickly. 

The data-driven approach worked so 
well that through much of our devel- 
opment, THIEF and SYSTEM SHOCK 2 
(two very different games) used the 
same executable and simply chose a 
different object hierarchy and data set 
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team believed in and achieved this 
vision, and special credit goes to audio 
designer Eric Brosius. 

As an element of the design, sound 
played two roles in THIEF. First, it was 
the primary medium through which 
the Als communicated both their loca- 
tion and their internal state to the 
player. In THIEF we tried to design Als 
with a broader range of awareness than 
the typical two states that Als exhibit: 
“oblivious” and “omniscient.” Such a 
range of internal states would be mean- 
ingless if the player could not perceive 
it, so we used a broad array of speech 
broadcast by the Als to clue in the play- 
er. While very successful for humanoid 
Als, we feel the more limited express- 
ibility of non-human creatures is the 
heart of why many customers didn’t 
like our “monster levels.” 

Second, the design used sounds gen- 
erated by objects in the game, especially 
the player, to inform Als about their sur- 
roundings. In THIEF, the Als rarely 
“cheat” when it comes to knowledge of 
their environment. Considerable work 
went into constructing sensory compo- 
nents sufficient to permit the Als to 
make decisions purely based on the 
world as they perceive it. This allowed 
us to use player sounds as an integral 


player could expect nearby Als to 
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become alarmed by the sound of com- 
bat or cries for help, and was thus 
encouraged to ambush opponents as 
quietly as possible. 

[In order for sound to work in the 
game as designed, we needed to imple- 
ment a sound system significantly 
more sophisticated than many other 
games. When constructing a THIEF mis- 
sion, designers built a secondary 
“room database” that reflected the 
connectivity of spaces at a higher level 
than raw geometry. Although this was 
also used for script triggers and Al 
optimizations, the primary role of the 
room database was to provide a repre- 
sentation of the world simple enough 
to allow realistic real-time propagation 
of sounds through the spaces. Without 
this, it is unlikely the sound design 
could have succeeded, as it allowed the 
player and the Als to perceive sounds 
more as they are in real life and better 
grasp the location of their opponents 
in the mission spaces. 

Focus, Focus, Focus. Early on, the 

@ Tuer plan was chock full of 

features and metagame elements: lots 
of player tools and a modal inventory 
user interface to manage them; multi- 
player cooperative, death-match and 
“Theft-match” modes; a form of player 
extra-sensory perception; player capaci- 
ty to combine world objects to create 
new tools; and branching mission 
structures. These and other “cool 
ideas” were correctly discarded. 

Instead, we focused in on creating a 
single-player, linear, mission-based 
game centered exclusively around 
stealth, with a player toolset that fit 
within the constraints of an extension 
of the QUAKE user interface. The notion 
came into full force with two decisions 
we made about seven months before 
we shipped. First, the project was 
renamed THIEF from the working title 
“The Dark Project,” a seemingly minor 
decision that in truth gave the team a 
GAME DEVELOPER 
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concrete ideological focus. Second, we 
decided preemptively to drop multi- 
player support, not simply due to 
schedule concerns, but also to allow us 
as much time as possible to hone the 
single-player experience. In the end, 
some missions didn’t achieve the 
stealth focus we wanted, particularly 
those originally designed for “Dark 
Camelot,” but the overall agenda was 
the right one. 
OBJECTIVES AND DIFFICULTY. One of 

@ the THIEF team’s favorite games 
during development was GOLDENEYE on 
the N64. We were particularly struck 
by the manner in which levels of diffi- 
culty were handled. Each level of diffi- 
culty had a different overlapping set of 
objectives for success, and missions 
were subtly changed at each level in 
terms of object placement and density. 
Relatively late in the development of 
THIEE, we decided such a system would 
work well in our game. Extending the 
concept, we added a notion that as dif- 
ficulty increased, the level of toleration 
of murder of human beings decreased. 
We also allowed players to change 
their difficulty level at the beginning of 
each mission. The system was a success 
in two ways. First, it made clear to the 
player exactly what “difficulty” meant. 
Second, it allowed the designers to cre- 
ate a very different experience at each 
level of difficulty, without changing 
the overall geometry and structure of a 
mission. This gave the game a high 
degree of replayability at a minimum 
development cost. 

MULTIPLE NARROW-PURPOSE SCRIPT- 

@ ING sotutions. Although the 
Object System provided a lot of flexibil- 
ity, we also needed a scripting language 
to fully specify object behaviors. Rather 
than create a single all-encompassing 
scripting system, we chose to develop 
several more focused tools for script- 
ing. This tiered scripting solution 
worked well. 





In creating our core “high-end” 
object scripting technology, we want- 
ed to allow designers with moderate 
programming skill to create complex 
object behaviors easily. Scripts were 
event-driven objects attached at run 
time to game objects, and contained 
data, methods, and message handlers. 
The game provided a selection of ser- 
vices to allow the script to query the 
world state and the game object state, 
and also to perform complex tasks. 
Our goal was to create a scripting lan- 
guage that offered source-level debug- 
ging, was fast, and was dynamic. The 
solution was essentially C++ in .DLLs, 
compiled by the C++ compiler, using a 
combination of classes and preproces- 
sor macros to ease interface publish- 
ing, handle dynamic linking, and pro- 
vide designers a clear programming 
model. Though used by both program- 
ming-savvy designers and program- 
mers, the fact that it was a real pro- 
gramming language prevented 
widespread use by all of the designers. 

Most designers were interested in 
customizing AI behaviors. For the Al 
we created a simpler scripting system, 
“Pseudo-scripts,” that were implement- 
ed as properties within the Object 
System. Pseudo-scripts took the burden 
of coding scripts off of the designers. 
The AI provided a stock set of triggers, 
such as “I see the player near an 
object” or “I see a dead body”; the 
designer provided the consequence of 
the trigger. Each Pseudo-script was 
edited in a dialog box presenting para- 
meters to tweak the “if” clause of the 
trigger, and space for a list of simple, 
unconditional actions to perform 
when the trigger fired. In this way, the 
custom behavioral possibilities of the 
Al at any moment were described by 
the aggregate of Pseudo-scripts that 
were attached to that AI. This approach 
had three benefits. First, it was simple 
enough so that designers with no pro- 
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gramming experience were comfort- 
able using it. Second, it narrowed the 
range of triggers a designer could use to 
a good pre-selected set, rather than giv- 
ing them an open-ended system that 
might not have worked as well. Finally, 
when and how to evaluate Al triggers, a 
potential run-time expense if not care- 
fully constructed, could be custom 
built by a programmer. 

The final scripting system built into 
THIEF was the Tagged Schema system. 
When the game required motions and 
sounds, it requested them as concepts 
modified by optional qualifiers, rather 
than directly. For example, an AI who 
had just heard the player would 
request the concept “moderate alert,” 
qualified with an optional tag like 
“+sense:sound.” A potential set of 
resources was then chosen using a pat- 
tern matcher; in this example, it would 
choose all samples in that Al’s voice 
expressing a generic “something’s not 
right,” all samples expressing “I heard 
something fishy,” but no samples 
expressing “I saw something fishy.” 
From this set, the specific resource was 
then chosen using a weighted random 
selection. The tables used were speci- 
fied by the designers using a simple 
language. Specifying motion and 
sound selection this way, designers cre- 
ated an interesting variety of random- 
ized environments and behaviors with- 
out changing the code of the game. 


What Went Wrong 


TROUBLE WITH THE Al. If one thing 

@ could be called out as the reason 
THIEF’s gameplay didn’t come together 
until late in the process, it would be 
the AI. The Al as a foil to the player is 
the central element of THIEF, and the AI 
we wanted wasn’t ready until late in 
the spring of 1998. As lead programmer 
and author of the final Al, I take full 
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responsibility for that. 

The original AI for THIEF was 
designed by another programmer 
before the requirements of the revised 
stealth design were fully specified. Six 
months after it was begun, the project 
director and overseer of the system left 
the team, and the most of the program- 
ming staff was temporarily reassigned 
to help ship another game that was in 
trouble. During the following months, 
development on that AI continued 
without any oversight and without a 
firm game design. Soon after, the pro- 
grammer working on the AI also left. 
While the core pathfinding data struc- 
tures and algorithms were basically 
sound, the code that generated the 
pathfinding database was extremely 
buggy. The design of the AI decision 
process was geared towards an action 
fighting game requiring little designer 
customization, rather than a stealth 
game that needed much more cus- 
tomization. Even worse, the high-level 
decision process in the AI had drifted 
away from a rigorous design and the 
code was extremely brittle. The whole 
situation was a disaster. 

These might not have been serious 
issues, except for one key mistake: | 
didn’t realize the depth of the problem 
quickly enough, and despite concerns 
expressed by programmer/designer 
Doug Church, I didn’t act fast enough. 
[ think highly of the programmer 
involved with the initial Al and wanted 
to avoid the natural but often misguid- 
ed programmer reaction within myself 
that I should just rewrite it my way. So, 
[ took the position that, while buggy, 
the system as a whole was probably 
sound. Several months and many 
sleepless nights later, I concluded that I 
had been sorely mistaken. 

By November 1997, I had the basics 
of a new design and began working on 
it. But all work had to stop in order to 
pull together an emergency proof-of- 


concept demo by the end of December 
to quell outside concerns that the team 
lacked a sound vision of the game. This 
turned into a mid-January demo, fol- 
lowed by an early February publisher 
demo, followed by a late February 
make-or-break demo. During this time 
the only option was to hack features as 
best we could into the existing AI. 
While better than losing our funding, 
constructing these demos was not good 
for the project. 

In the end, work on the new AI didn’t 
begin until mid-March. Despite the fact 
that our scheduled ship date was just six 
months away, we threw away four-fifths 
of our existing AI code and started over. 
After a hair-raising twelve-week stretch 
of grueling hours, the AI was ready for 
real testing. Had I committed to a 
rewrite two months earlier the previous 
autumn, I believe the AI would have 
been ready for real use three to five 
months sooner. 

AN UNCERTAIN RENDERER. [he pro- 
@ ject was started because of the 
renderer, rather than the reverse. The 
basic core of the renderer for THIEF was 
written in the fall of 1995 as an after- 
hours experiment by programmer Sean 
Barrett. During the following year, the 
renderer and geometry-editing tools 
were fleshed out, and with “Dark 
Camelot” supposed to ship some time 
in 1997, it looked like we would have a 
pretty attractive game. Then, at the 
end of 1996, Sean decided to leave 
Looking Glass. Although he periodical- 
ly contracted with us to add features, 
and we were able to add hardware sup- 
port and other minor additions, the 
renderer never received the attention it 
needed to reach the state-of-the-art in 
1998. The possibility that we might not 
have a point programmer for the ren- 
derer weighed heavily on the team. 
Fortunately, Sean remained available 
on a contract basis, and other members 
of the team developed sufficient 
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POSTMORTEM 


One of the featureed weapons is the fire arrow. 





knowledge of the renderer so that we 
shipped successfully. In the end, we 
shipped a renderer appropriate for our 
gameplay, but not as attractive as other 
high-profile first-person titles. 

This may prompt the question of 
why we didn’t simply license a render- 
er. When the project started a few 
months into 1996, the avalanche of 
QUAKE licenses hadn’t really begun 
and UNREAL was still two years away. 
By the time licensing was a viable 
choice, the game and the renderer 
were too tightly integrated for us to 
consider changing. 

& LOSS OF KEY PERSONNEL AMID CORPO- 
@ RATE ANGST BEYOND OUR CONTROL. 
Midway through 1997, THIEF was just 
starting to gather momentum. We 
were fully staffed and the stealth design 
was really starting to get fleshed out. 
Unfortunately, Looking Glass’s finan- 
cial situation was bleak. Few emotions 
can compare to the stress of heading to 
work not knowing who might be laid 
off, including yourself, or whether the 
doors would be locked when you got 
there. The company shed half of its 
staff in a span of six months, and while 
the active teams tried to stay focused, it 
was hard when one day the plants were 


gone, another day the coffee machine, 


then the water cooler. 

Some of the THIEF team couldn’t 
continue under these conditions. We 
lost two programmers, including the 
former lead programmer, and a 
designer. When we were forced to 
close our Austin office, we lost our 


producer, Warren Spector, as well as 


some programmers who made valu- 
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able technology 
contributions to 
our engine. All of 
these individuals 
are now on Ion 
Storm’s DEUS EX 
team. Although it 
took some 
months to fully 
restore the spirit 
of the rest of the 
team, we held 
together and the 
company eventu- 
ally rebounded. 
Perhaps it 
bestowed a sto- 
icism that comes 
from knowing 
that however bad 
things might seem, you’ve already seen 
worse. 
UNDERVALUED EDITOR. One of the 
@ boils never lanced on the pro- 
ject was our editor, Dromed. Although 
it was sufficiently powerful and pro- 
vided the essential functionality we 
needed to ship the game, Dromed was 
a poorly documented and sometimes 
disagreeable editor. Dromed was first 
developed as a demonstration editor 
when the target platform of the game 
was DOS. As a demo, it never received 
the kind of formal specifications and 
designs one would expect for the cen- 
tral experience of the design team. As a 
DOS application, it lacked the consis- 
tent and relatively easy-to-use user- 
interface tools of Windows. An early 
mistake was our failure to step back 
and formally evaluate the editor, and 
then rebuild it based on our experi- 
ence constructing the demo editor. We 
also should have designed a proper 
editor framework, and hired a dedicat- 
ed Windows user-interface program- 
mer to support it through develop- 
ment. In retrospect, the time lost 
cleaning up the editor probably would 
have been saved on the back end of 
the project. 
INADEQUATE PLANNING. Although it 
@ is a cliché in the software 
industry to say our scheduling and 
budget planning were woefully inade- 
quate, the THIEF project suffered greatly 
from this malady. There were several 
elements to our deficient planning. 
During “Dark Camelot,” and continu- 
ing through the first half of THIEF, we 
staffed the team before the design and 





technology was sufficiently mature. In 
THIEF, this led us to rush towards finish- 
ing the design, when we didn’t necessar- 
ily understand the design and technolo- 
gy. With insufficient specifications of 
both the code systems and mission 
designs, we ended up doing lots of con- 
tent that was essentially wrong for the 
game we were making. Code was written 
and spaces were built that weren’t well 
directed towards the goals of the project. 

To make matters worse, we failed to 
reassess core scheduling assumptions 
carefully once the schedule began to 
slip. Captives of a series of unrealistic 
schedules, we didn’t leave enough 
time for the sort of experimentation, 
dialogue, and prototyping a project 
like THIEF needs. Late in the winter of 
1998, many of our scheduling mis- 
takes had been corrected. Still, during 
the remainder of the project, the lega- 
cy of our earlier missteps required cut- 
ting missions that relied on technolo- 
gy we didn’t have, and reworking 
missions not focused on the core 
gameplay. 


Stepping Back from the Project 


TO was constructed as a set of 
appropriately abstract reusable 
game components designed for creat- 
ing object-rich, data-driven games. 
Although increasing the cost of devel- 
opment, this approach allowed 
Looking Glass to leverage various tech- 
nologies across disparate types of 
games, from the first-person action 
game SYSTEM SHOCK 2 to our combat 
flight-simulator FLIGHT COMBAT. In our 
next-generation technology, some of 
the systems, such as the AI and the 
Object System, will merely be revised, 
not rewritten. We intend to continue 
with this development philosophy in 
our future games. 

The next time around, our approach 
to constructing the engine will differ. 
The engine will be scheduled, staffed, 
and budgeted as a project in its own 
right. The editor will be treated as more 
of a first-class citizen than was the case 
in THIEF. Finally, a content develop- 
ment team will not be geared up until 
the technology is sufficiently mature to 
allow for an informed game design 
process. 

Oh, and we’ll get our schedules 
right — really. & 
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Continued from page 64. 

here on planet Earth are also responsible 
for the other 80 percent of a game’s 
audio, which includes the dialogue, 
sound effects, interface feedback, and 
so on. The longer we promote our- 
selves as musicians, the harder it’s 
going to be for us to work our way into 
the upper ranks of the team, which is 
where we must be if we are going to 
push audio design through to the next 
level. We need to be technologists, 
innovators, and sonic architects, not 
bong-smoking, noodle-headed, elec- 
tronic music geeks. 

THE HOLLYWOOD Bic WiG. There is also a 
large number of linear media post- 
production sound designers who work 
amongst us. While many of these profes- 
sionals are extremely talented when it 
comes to complementing a fixed asset, 
like a film, with sound effects, dialogue 
and music, they have limited expertise in 
creating an interactive soundscape. 
Programmers spend a great deal of time 


Thanks, 


Dear Game Developer, 


If you’re technically astute and have a way with words, 
Game Developer magazine needs you. We’re looking for 
feature articles on graphics programming, testing and QA, 
producer/management issues, audio, art and animation, 
internet game development, new technologies, and game 
design. We’re also looking for people to review game : 
development tools. 


thinking universally about all aspects of 
the world they will be creating, and we 
also need to think well beyond the sonic 
content we create. 


There was at least one GDC session per 
day that was sprinkled with a heaping 
dash of crow about how the “sound 
design didn’t turn out like I wanted it 
to.” The audio professionals always point 
their fingers at the programmers and pro- 
ducers for this shortcoming, which 
seems like a pathetic cop-out to me. 
When was the last time you provided a 
code sample with your spec? Every piece 
of media on your list should have a com- 
plete description of how it interacts with 
all of its dependencies: user input, artifi- 
cial intelligence, and so on. Leave no 
stone unturned, do your damn home- 
work, take control of your world because 
if you don’t, no one will. Waxing ecstatic 
about design concepts without being 
able to provide the developer with con- 
crete examples of how it can be imple- 
mented is negligent and can be offensive 


Please send your abstract or outline to: 
adunne@compuserve.com 


Writer’s guidelines can be downloaded from our web site at: 
http: //www.gdmag.com/writguid. htm 


Ma Yo 


Editor in Chief 
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to someone who has spent the last year 
thinking about everything but the audio. 
Educate yourself on basic programming 
concepts, understand the relationship 
between the sound in the game and 
everything it touches, and be a diplomat 
when it comes to working with the rest 
of the team, because they are holding 
your design in their hands. 

I believe that we are only as influential 
as we want to be, because audio program- 
ming can be the most emotional and 
subliminal (read: core) information a 
videogame can deliver. However, if we 
continue to play the role of the “audio 
guy,” we will constantly be ruled by pro- 
gram managers, producers, and develop- 
ers, who all consider themselves to be 
critical to the product’s success. If you 
start showing up to the table with the 
right cards in your hand, I can guarantee 
that your budget will grow, you'll break 
more bread, and the end product will 
bear a much greater resemblance to your 
original version. & 
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nursery rhyme, except that I was not 
short and stout, I was hot, steaming and 
pissed off. At one of the audio sessions, 
a well-respected audio designer began 
his presentation in a room full 

of budding young produc- 
ers with the classic, “I’m 
an audio guy and get 
no respect in this 
industry” inferiority 
complex. I’ve caught 
myself falling into 
this same trap in 
meetings and 
such, but 
propagating 
this 


















impression 
throughout the 
gaming community is a big mis- 
take. We need to be proactive 

contributors to our community. 


We need to assert confidence and power 


if we are going to be respected as equals. 
There are three major stereotypes 

that are keeping us down, yet we regu- 

larly embody them in public: 

Tue “Aupio Guy.” [ am sick of people 

referring to themselves as the “audio 

guy.” When I think of an “audio guy,” I 





Don’t Call Me “Audio Guy” 


hile attending the recent Game 
Developers’ Conference in San 
Jose, I left each audio session feel- 


ing like the little teapot in that 


think of someone who crawled off of 
the Van Halen 1984 tour. After destroy- 
ing the remainder of his hearing doing 
live sound to support his cocaine habit, 
he landed a “day job” in the videogame 
industry. As you laugh, keep in mind 
that these people do 


\ \ exist in 
our industry and they are 
all around you. (Oh, and by the way, 
we're not all guys. While the traditional 
linear media post-production scene has 
been a pickle convention for some time 
now, the videogame industry contains 
some of the most talented female audio 
professionals in the world.) 


by Matt Johunaton 


Stee 


“tig 











Association, Matthew Lee Johnston is now a Senior Audio Designer at Microsoft. He 





can be reached at mattj@microsoft.com 
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At a GDC audio session, I asked a 
question that made some people 
uncomfortable. | wanted to know why 
all current audio tools are designed with 
control surface metaphors that were 
innovated in or before the late 1950s 
(such as volume sliders, piano rolls, 
knobs, and VU meters). An “audio guy” 
actually had the audacity to tell me after 
the session that “you young guys just 
need to sit back and wait your turn.” | 
think that his main fear was that he will 
someday have to become part program- 
mer and part audio guy. The truth is, he 
better, because the videogame of the 
future will demand complex and innov- 
ative audio systems that can’t be 
designed by mere “code guys,” 
because they don’t understand 
the subtleties of audio design like 
he does. But unless he knows 
how to implement those ideas 
himself, he will be back 

behind the mixing board 
faster than you Can say, 

“More vocals in the 

monitor please.” 
THE Musician. At another ses- 
sion, one well known fig- 
ure within our community 

(who referred to himself as 

“The Almighty”) 

—— addressed a room of 
more than 100 audio profession- 
als as “musicians.” I’m all for 

making music, but let’s face it, 
music is only a small part of what 
goes into a game these days. Now, | 
understand that there are many people 
who represented themselves to their 
current employers as “computer 
experts” because they were wizards with 
the musician’s equivalent to a writer’s 
word processor, the MIDI sequencing 
program. | also understand that some 
people fancy themselves rock stars 
because they make a good living by 
orchestrating hundreds of little black 
boxes to do exactly what they want, 
when they want them to, and crank out 

the epic soundtrack to “Syncopation 2, 

The Aural Equivalent to Chinese Water 

Torture.” However, most of us down 







Continued on page 63. 


http://www.gdmag.com 








Nation’s Largest 
Motion Capture 
Service Studios 


/ F Now Open From 
~ Coast to Coast 


° Plug-ins to Alias, 3D Studio MAX, Softimage, Lightwave, Maya ¢ 3D Face 
Captures * Optical 10 Camera Systems © Animals of Varied Proportions 
¢ Real-Time Viewing of Character Animation Multiple Person Captures 


Los Angeles 





New York 

Performance , aa ; 
Capture . MP oder Uprising Stuchos 
Studios fr 

Refreshing Creativity « State of the Art Studio and Our services fill the gap between raw data 

Equipment ¢ Large Volume, Multiple Actor/Animal and final animation! Our staff has experi- 

Capture ¢ Real Time Optical Capture * Simultaneous ence capturing over 10,000 moves used for 

3D Face and Full-body Captures games, broadcast and feature films. 

Services: Services: Facility: 

¢ Motion editing ° Training ¢ Motion editing ¢ 5,800 square feet 

¢ Motion re-targeting ° Project management ¢ Data re-scaling Studio space 

¢ Model attachment ¢ Consultation ¢ Skin attachment ¢ Large capture space 

¢ Software tools ° Portable system e Data conversion e Portable system 

¢ Cyberscan available ¢ Project consulting 

11872 LaGrange, Los Angeles, CA 90025 USA Brooklyn Navy Yard, NY USA 

310-704-3036 FAX 310-841-2076 /18-852-0811 FAX 718-858-2459 

www. performancecapture.com www. modernuprising.com 

«38 MotionAnalysis MoCap Systems also used by these industry leaders: 


Acclaim Entertainment — Brilliant Digital Entertainment CAPCOM Electronic Arts — Eolith 
Gremlin Koei Konami Namco RARE Riverhillsoft SNK Sony Square 


The Best in Game Development Technology! 







































Bink - the new true-color codec from the author of Smacker! 
Bink is available and revolutionizing game videos just as Smacker did four years ago! Bink is a “better-than-MPEG-II” class 
codec. Yes, that’s right - better than DVD! It produces higher quality than MPEG II and is up to three times faster than other 


software decoders. Now there is finally a true-color codec good enough for game developers. 


Bink is a hybrid block-transform and wavelet codec that can encode your video using 16 different compression techniques 
(wavelet, DCT, motion compensation, a variety of vector quantizers, Smacker-style, etc). With all of these techniques in one 
codec, Bink can handle almost any type of video. Bink also includes a psycho-acoustic based audio codec that is capable of 


8 to 1 perceptibly lossless compression, so your audio will sound as good as your video looks. 





Bink is available now for Windows 95/NT (DOS and MacOS coming soon) and supports DirectDraw, DIBSections, DirectSound, and the Windows 
waveOut system. Bink uses the YUV colorspace, so it can use DirectDraw overlays for hardware color conversion and smooth scaling. Bink also includes a 
bunch of hand-optimized assembly YUV to RGB colorspace blitters, so you'll be able to access Bink’s output in any RGB format you like. Bink is a high-end 
codec, and as such, it requires at least a Pentium/150 (Pentium/200 preferred). Bink really doesn’t have a minimum CD requirement - 320x240 animations 
look great at 150 kps (Ix CD) and 640x480’s only need 450 kps (3x CD). Bink does need a reasonable video card that is capable of pumping out all those 
true-color pixels, though. The Bink SDK is very similiar to our popular Smacker SDK, so integration is easy, and upgrading from Smacker is easier still. 


Youre really going to love Bink - download the Bink Tools (or call for a demo CD) and try it on your videos today! 






MPEG Layer-3 decompression support! 
Miles 5.5 includes both Intel x87 and AMD 3DNow optimized MPEG Layer-3 decoders (patent rights licensed from 
Thomson and Fraunhofer). MP3 provides perceptually lossless compression at 11 to 1 - your sound files will be more than 
ten times smaller and will sound exactly the same! Miles supports both on-the-fly decompression (for minimal memory 
use), or pre-playback decompression (for minimal CPU hit). Miles can even MP3-compress DLS instrument data. 





High-level 3D sound support including built-in RSX 3D technology! 
The Miles high-level 3D sound API supports our built-in RSX 3D technology (acquired from Intel), fast 2D simulated, 
SOUND SYSTEM DirectSound3D, Dolby Surround, Creative's EAX, and Aureal’s A3D (all selectable at run-time). Better still, you won't be 


tied toa particular low-level 3D API, so you can use the technology that makes your game sound best. 


New digital sub-system! Per-sample reverb, input API with chat codec, faster mixing, filtering, and plug-in support! Replaces the DirectSound mixer! 
Miles 5.5 has tons of new digital audio features: per-sample reverb, input support (including a low-bandwidth voice chat codec), on-the-fly filtering, and a 
new plug-in system for adding run-time sound effects which can be called at the pre- or post-mix stage. Miles also uses its own mixer even when running 


under DirectSound - if your game starts and stops multiple sound effects frequently, then this new feature can potentially double your frame rate! 


Of course, Miles 5.5 provides all of the features you loved in earlier versions: complete digital mixing and format conversions, interactive MIDI playback 


with a built-in DLS software synthesizer, hard drive or CD-ROM digital streaming, red book CD-audio support, powerful callbacks, and much more! 








Smacker - the best 256-color codec on the planet! 
Smacker is still available and better than ever! Smacker is still the best codec for situations such as: 256 color games (of 
course), games targeting low-end machines (Pentium 133 and below), games that need video sprites or video transparency 
(which is much faster in 256 colors), cell-based (cartoon-style) videos, animated 3D texture compression, and very high- 


resolution videos (800x600 or 1024x768 - only Smacker is fast enough for videos this big). 


The Smacker SDK is available for DOS, 16-bit Windows, Windows 95/ NT, Win32s, 68K Mac, and PowerMac. The 


SDK’s API is identical across these platforms, and Smacker files can be played without conversion on each platform. 





Smacker supports DirectDraw, DIBSections, WinG, DispDIB, SVGA VESA, and GWorlds (MacOS) for graphics output. For sound output, Smacker 
supports the Miles Sound System, DirectSound, the Windows waveOut system, and the Sound Manager (MacOS). 


